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FOREWORD 


The  project  (Grant  No.  W91  INF-06-01-0469)  titled  Formal-language-theoretic 
Control  &  Co-ordination  of  Mobile  Robots  was  started  on  September  20,  2006  and 
ended  on  September  19,  2007.  It  was  conducted  under  the  leadership  of  Prof.  Asok  Ray 
and  Dr.  Ishanu  Chattopadhyay,  Pennsylvania  State  University,  University  Park,  PA.  This 
research  project  has  developed  a  novel  approach  to  control  of  co-operating  and  non-co¬ 
operating  teams  of  autonomous  and  semi-autonomous  agents  and  has  made  fundamental 
contributions  to  the  enhancement  of  the  state  of  the  art  in  the  field  of  Probabilistic 
Robotics.  On  the  experimental  side  of  the  project,  Penn  State  has  developed  the 
Networked  Robotics  and  Sensors  Laboratory  (NRSL)  for  conducting  research  in  robotics, 
which  has  been  supported  through  a  DURIP  equipment  grant. 

The  research  work,  conducted  under  this  project,  has  resulted  in  8  scholarly 
publications.  This  project  has  laid  the  groundwork  for  effectively  transferring  the  newly 
developed  technology  to  defense  industry  through  future  SBIR  and  STTR  projects  in 
collaboration  with  industry. 
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STATEMENT  OF  THE  PROBLEM  STUDIED 


The  Future  Combat  Systems  (FCS)  program  of  the  U.S.  Army  calls  for  collaboration 
among  heterogeneous  groups  of  semi-autonomous  and  autonomous  mobile  platforms, 
such  as  Unmanned  Ground  Vehicles  (UGVs)  and  Unmanned  Aerial  Vehicles  (UAVs), 
supported  by  communication  of  onboard  sensor  and  ancillary  information  among 
individual  platforms  and  human  users.  The  research  conducted  under  this  project 
addresses  the  challenge  of  developing  C4/  systems  for  applications  to  groups  of 
autonomous  vehicles  that  must  adapt  themselves  to  operate  in  uncertain  and  dynamically 
evolving  environments  of  battlefields.  The  research  has  formulated  and  experimentally 
validated  robust  adaptive  algorithms  and  software  codes  for  decision  &  control  of  mobile 
robotic  platforms,  as  applied  to  real-time  computation  and  execution  of  combat  mission 
strategies.  These  algorithms  are  executable  within  a  general-purpose  programming 
language  environment  and  make  use  of  the  generative  power  of  formal-language- 
theoretic  models  instead  of  ad-hoc  rule-based  expert  systems.  Research  efforts  have  been 
concentrated  in  the  following  inter-related  areas. 

•  Formulation  of  operational  intelligence  models  in  a  formal-language-theoretic 
setting  for:  (i)  autonomous  agents  and  decision  &  control  architectures,  and  (ii) 
interchangeable  (i.e.,  platform-independent)  human-machine  interactions 

•  Algorithm  development  for  intelligent  coordination  of  autonomous  agent  teams  to 
accomplish  complex  mission  tasks  with  automated  resource  and  risk  optimization 
with  (possibly)  incomplete  knowledge  of  work-space  parameters  due  to:  (i) 
insufficient  information,  and  (ii)  sensor  and/or  communication  link  failures 

•  Experimental  validation  of  the  mathematical  tools  and  software  codes  in:  (i) 
Systems  Simulation  Laboratory,  and  (ii)  Networked  Robotics  &  Sensors 
Laboratory,  developed  at  Penn  State  under  ARO-funded  MURI  and  DURIP  grants 
(PM:  Dr.  M-H.  Chang) 
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SUMMARY  OF  THE  MOST  IMPORTANT  RESULTS 


Formal-language-theoretic  Control 
& 

Co-ordination  of  Mobile  Robots 

Principal  Investigator:  Professor  Asok  Ray,  Pennsylvania  State  University 
Key  Contributor:  Dr.  Ishanu  Chattopadhyay,  Pennsylvania  State  University 

The  verifiable  outcomes  of  the  project  include  novel  algorithms,  associated  numerical 
methods,  and  validated  software  codes  for  decision  &  control  of  a  group  of  mobile 
robots,  focusing  on  the  following  issues. 

•  Coordination  of  heterogeneous  groups  of  autonomous  mobile  platforms 

•  Robustness  of  controlled  behavior  under  disturbances  and  loss  of  information 

•  Computational  speed  and  memory  requirements  for  real-time  on-board  execution, 
especially  under  unusual  off-nominal  circumstances 

Furthermore,  the  study  of  robust  optimal  control  and  coordination  of  mobile  robots 
carried  out  under  the  project  will  enhance  C4/  capabilities  of  DoD  for  completely 
autonomous  or  command-aided  remote  mission  execution  in  uncertain  and  dynamically 
evolving  combat  environments.  The  research  project  has  pursued  technology  transition 
and  develop  multidisciplinary  educational  and  training  programs  for  academia  and 
industry,  which  are  vital  to  DoD. 

On  the  theoretical  side  of  the  project,  a  rigorous  methodology  has  been  developed  for 
automated  modeling  of  observed  behavior  of  autonomous  agents  and  designing  optimal 
mission  strategies  via  a  novel  measure-theoretic  optimization  of  probabilistic  finite  state 
behavior  generators.  The  past  decade  has  witnessed  the  development  of  a  range  of 
methodologies  for  design  and  control  of  autonomous  systems,  ranging  from  model-based 
to  purely  reactive  paradigms.  One  of  these  approaches,  Probabilistic  Robotics,  has  led  to 
implementations  with  significant  autonomy  and  robustness.  The  research  carried  out 
under  this  project  enhances  the  state  of  the  art  in  Probabilistic  Robotics  by  addressing  the 
following  key  issues: 

•  Probabilistic  Perception:  Robots  are  inherently  uncertain  about  the  state  of  their 
environments.  Uncertainty  arises  from  sensor  limitations,  noise,  and  the  fact  that  most 
interesting  environments  are,  to  a  certain  degree,  unpredictable.  Moreover,  a 
probabilistic  robot  knows  about  its  own  ignorance  -  a  key  prerequisite  of  true 
autonomy. 

•  Probabilistic  Control:  Autonomous  robots  must  act  in  the  face  of  uncertainty. 
Probabilistic  decision-making  approaches  take  the  robot’s  uncertainty  into  account; 
some  consider  only  the  robot’s  current  uncertainty,  others  anticipate  future 
uncertainty. 
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To  appear  in  International  Journal  of  Control 

Structural  Transformations  of  Probabilistic  Finite  State  Machines* 


Ishanu  Chattopadhyay* 

ixcl28@psu.edu 


Asok  Rayt 

axr2@psu . edu 


Abstract — Probabilistic  finite  state  machines  have  re¬ 
cently  emerged  as  a  viable  tool  for  modeling  and  analy¬ 
sis  of  complex  non-linear  dynamical  systems.  This  paper 
rigorously  establishes  such  models  as  finite  encodings  of 
probability  measure  spaces  defined  over  symbol  strings.  The 
well  known  Nerode  equivalence  relation  is  generalized  in 
the  probabilistic  setting  and  pertinent  results  on  existence 
and  uniqueness  of  minimal  representations  of  probabilistic 
finite  state  machines  are  presented.  The  binary  operations  of 
probabilistic  synchronous  composition  and  projective  com¬ 
position  are  introduced,  which  have  applications  in  symbolic 
model-based  supervisory  control  and  in  symbolic  pattern 
recognition  problems.  The  results  are  elucidated  with  nu¬ 
merical  examples  and  are  validated  on  experimental  data  for 
statistical  pattern  classification  in  a  laboratory  environment. 


Index  Terms — Formal  Language  Theory;  Probabilistic  Fi¬ 
nite  State  Automata;  Minimization  of  Probabilistic  Au¬ 
tomata;  Model  Order  Reduction 

1.  Introduction  &  Motivation 

Probabilistic  finite  state  machines  have  recently  emerged 
as  a  modeling  paradigm  for  constructing  causal  models  of 
complex  dynamics.  The  general  inapplicability  of  classical 
identification  algorithms  in  complex  non-linear  systems  has 
led  to  development  of  several  techniques  for  construction 
of  probabilistic  representations  of  dynamical  evolution  from 
observed  system  behavior.  The  essential  feature  of  a  ma¬ 
jority  of  such  reported  approaches  is  partial  or  complete 
departure  from  the  classical  continuous-domain  modeling 
towards  a  formal  language  theoretic  and  hence  symbolic 
paradigm  [1][2].  The  continuous  range  of  a  sufficiently  long 
observed  data  set  is  discretized  and  tagged  with  labels  to 
obtain  a  symbolic  sequence  [2],  which  is  subsequently  used 
to  compute  a  language-theoretic  finite  state  probabilistic 
predictor  via  recursive  model  update  algorithms.  Symboliza¬ 
tion  essentially  discretizes  the  continuous  state  space  and 
gives  rise  to  probabilistic  dynamics  from  the  underlying 
deterministic  process,  as  illustrated  in  Fig.  |T| 

Among  various  reported  symbolic  reconstruction  algo¬ 
rithms,  Causal-State  Splitting  Reconstruction  (CSSR)  [1] 
computes  optimal  representations  (e.g.,  e-machines)  and  is 


*This  work  has  been  supported  in  part  by  the  U.S.  Army  Research 
Office  under  Grant  Nos.  W911NF-06-1-0469  and  W911NF-07-1- 
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Fig.  1.  Emergence  of  probabilistic  dynamics  from  the  underlying  de¬ 
terministic  system  due  to  discretization  as  symbolic  states  171,(72, <73! 
and  a,  T  are  symbolic  events 


reported  to  yield  the  minimal  representation  consistent  with 
accurate  prediction.  In  contrast,  the  D-Markov  construc¬ 
tion  [2]  produces  a  sub-optimal  model,  but  it  has  a  significant 
computational  advantage  and  has  been  shown  to  be  better 
suited  for  online  detection  of  small  parametric  anomalies  in 
dynamic  behavior  of  physical  processes  [3]. 

This  paper  addresses  the  issue  of  structural  manipulation 
of  such  inferred  probabilistic  models  of  system  dynamics.  The 
ability  to  transform  and  manipulate  the  automaton  structure 
is  critical  for  design  of  supervisory  control  algorithms  for 
symbolic  models  and  real-time  pattern  recognition  from  sym¬ 
bol  sequences.  Specific  issues  are  delineated  in  the  sequel. 


A.  Applications  of  Symbolic  Model-based  Control 

The  natural  setting  for  developing  control  algorithms  for 
symbolic  models  is  that  of  probabilistic  languages.  The  no¬ 
tion  of  probabilistic  languages  in  the  context  of  studying 
qualitative  stochastic  behavior  of  discrete-event  systems  first 
appeared  in  [4]  and  [5],  where  the  concept  of  p-languages  (’p’ 
implying  probabilistic)  is  introduced  and  an  algebra  is  devel¬ 
oped  to  model  probabilistic  languages  based  on  concurrency. 
A  multitude  of  control  algorithms  for  p-language-theoretic 
models  have  been  reported.  Earlier  approaches  [6][7]  at¬ 
tempt  a  direct  generalization  of  Ramadge  and  Wonham’s 
Supervisory  Control  Theory  [8]  for  deterministic  languages 
and  proves  to  be  somewhat  cumbersome  in  practice.  A  signif¬ 
icantly  simpler  approach  is  suggested  in  [9]  [10]  [11],  where 
supervisory  control  laws  are  synthesized  by  elementwise 
maximization  of  a  language  measure  vector  [12][9]  to  ensure 
that  the  generated  event  strings  cause  the  supervised  plant 
to  visit  the  "good"  states  while  attempting  to  avoid  the  "bad" 
states  optimally  in  a  probabilistic  sense.  The  notion  of  "good" 
and  "bad"  is  induced  by  specifying  scalar  weights  on  the 
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model  states,  with  relatively  more  negative  weights  indicat¬ 
ing  less  desirable  states.  Unlike  the  previous  approaches,  the 
measure-theoretic  approach  does  not  require  a  "specification 
automaton";  however,  a  specification  weight  is  assigned  to 
each  state  of  the  finite  state  machine.  (Note:  These  states 
are  different  from  the  states  obtained  via  symbolic  recon¬ 
struction  of  observed  physical  data.) 

Figure  [2]  illustrates  the  underlying  concept.  The  symbolic 
model  shown  on  the  left  which  has  three  states  q2,  q2,  <?3,  while 
the  control  objective  is  specified  by  weights  +1  and  -1  on 
states  c\a,  cIb  of  the  two-state  automaton  on  the  right. 


Fig.  2.  Symbolic  Model  and  Control  Specification 


1/0.3 


Fig.  3.  Imposing  control  specification  by  probabilistic  synchronous 
composition  of  automata 

Recalling  that  every  finite  state  automaton  induces  a  right 
invariant  partition  on  the  set  of  all  possible  finite  length 
strings,  the  above  situation  is  illustrated  by  Figs.  [3ja)  and 
[3]b).  The  operation  of  probabilistic  synchronous  composition, 
defined  later  in  this  paper,  resolves  the  problem  by  consider¬ 
ing  the  product  partition  in  Fig.  [3{c).  Then,  the  given  model 
is  transformed  into  the  one  shown  in  Fig.  (3jd),  on  which 


the  optimization  algorithm  reported  in  [11]  can  be  directly 
applied  to  yield  the  optimal  supervision  policy. 

B.  Applications  of  Symbolic  Pattern  Recognition 

As  mentioned  earlier,  the  symbolic  reconstruction  algo¬ 
rithms  [1][2]  generate  probabilistic  finite  state  models  from 
observed  time  series.  However,  in  a  pattern  classification 
problem,  one  may  be  only  interested  in  a  given  class  of 
possible  future  evolutions.  For  example,  as  illustrated  in 
Fig.  ED  while  the  systems  Gi,G2,  , Ck  yield  different  sym¬ 
bolic  models  A1,A2/  ■  ■  ■  ,  Ak,  we  may  be  only  interested  in 
matching  a  given  template,  i.e.,  knowing  how  similar  the 
systems  are  as  far  as  strings  with  even  number  of  Os  is 
concerned  (Note:  =  {strings  with  even  number  of  Os)). 

The  operation  of  projective  composition,  defined  in  this  pa¬ 
per,  allows  transformation  of  each  model  A,  to  the  structure 
of  the  template  while  preserving  the  distribution  over  the 
strings  of  interest,  and  is  of  critical  importance  in  symbolic 
pattern  classification  problems.  As  shown  in  the  sequel,  the 
model  order  of  the  machines  A,  is  not  particularly  impor¬ 
tant;  hence  projective  composition  accomplishes  model  order 
reduction  within  a  quantifiable  error. 


C.  Organization  of  the  Paper 

The  paper  is  organized  in  seven  sections  including  the 
present  one.  Section  [2]  presents  preliminary  concepts  and 
pertinent  results  that  are  necessary  for  subsequent  develop¬ 
ment.  Section  [3]  introduces  the  concept  of  probabilistic  finite 
state  automata  as  finite  encodings  of  probability  measure 
spaces.  The  concept  of  Nerode  equivalence  is  generalized  to 
probabilistic  automata  and  the  key  results  on  existence  and 
uniqueness  of  minimal  representations  are  established.  Sec¬ 
tion  [4]  presents  metrics  on  the  space  of  probability  measures 
on  symbolic  strings  which  is  shown  to  induce  pseudometrics 
on  the  space  of  probabilistic  finite  state  automata.  Along 
this  line,  the  concept  of  probabilistic  synchronous  compo¬ 
sition  is  introduced  and  the  results  are  elucidated  with  a 
simple  example.  Section  [5]  defines  projective  composition 
and  invariance  of  projected  distributions  is  established.  A 
numerical  example  is  provided  for  clarity  of  exposition.  Sec¬ 
tion  [6]  demonstrates  applicability  of  the  developed  method 
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to  a  pattern  classification  problem  on  experimental  data. 
The  paper  is  summarized  and  concluded  in  Section  [7]  with 
recommendations  for  future  research. 

2.  Preliminary  Notions 

A  deterministic  finite  state  automaton  ( DFSA )  is  de¬ 
fined  [13]  as  a  quintuple  C,  =  (Q,  L,6,q,,Qm),  where  Q  is  the 
finite  set  of  states,  and  qi  e  Q  is  the  initial  state;  E  is  the 
(finite)  alphabet  of  events.  The  Kleene  closure  of  E,  denoted 
as  E*,  is  the  set  of  all  finite-length  strings  of  events  including 
the  empty  string  e;  the  set  of  all  finite-length  strings  of 
events  excluding  the  empty  string  e  is  denoted  as  E+  and  the 
set  of  all  strictly  infinite-length  strings  of  events  is  denoted 
as  E“.  A  subset  of  E"’  is  called  an  oj-language  on  the  alphabet 
E  and  a  subset  of  E*  is  called  a  *-language.  If  the  meaning 
is  clear  from  context,  we  refer  to  a  set  of  strings  simply  as 
a  language.  The  function  5  :  Q  x  E  — »  Q  represents  the  state 
transition  map  and  5*  :  Q  x  E*  — >  Q  is  the  reflexive  and 
transitive  closure  [13]  of  5  and  Qm  c  Q  is  the  set  of  marked 
(i.e.  accepting)  states.  For  given  functions  /  and  g,  we  denote 
the  composition  as  /  °  g. 

Definition  2.1:  The  classical  Nerode  equivalence  N  [13] 
on  E*  with  respect  to  a  given  language  L  is  defined  as: 

Vx,  y  e  E*  JxJVy  o  (Vu  e  E*  (  xu  e  L)  <=>  (yu  e  L)  (1) 

A  language  L  c  E*  is  regular  if  and  only  if  the  corresponding 
Nerode  equivalence  is  of  finite  index  [13]. 

Probabilistic  Finite  State  automata  (PFSA)  considered  in 
this  paper  are  built  upon  Deterministic  Finite  State  Au¬ 
tomata  (DFSA)  with  a  specified  event  generating  function. 
The  formal  definition  is  stated  next. 

Definition  2.2:  (PFSA)  A  probabilistic  finite  state  au¬ 
tomata  (PFSA)  is  a  quintuple  P,  =  (Q,  E,  b,  q,,  ft)  where  the 
quadruple  ( Q,L,5,qi )  is  a  DFSA  with  unspecified  marked 
states  and  the  mapping  ft  :  Q  x  E  — >  [0,1]  satisfies  the 
following  condition: 

IqjeQ,  YJMq„a)=\  (2) 

oeL 

In  the  sequel,  n  is  denoted  as  the  event  generating  function. 
For  a  PFSA  P,,  cardinality  of  the  set  of  states  is  denoted  as 
NUMSTATES(P;). 

Definition  2.3:  For  every  PFSA  Pi  =  (Q,  E,  5,  q,,  ft),  there 
is  an  associated  stochastic  matrix  n  e  irNumstates(p,)xNijmstates(P;)^ 
called  the  state  transition  probability  matrix,  which  is  defined 
as  follows: 

n;jh=  Yj 

We  further  note  that  for  every  stochastic  matrix  n,  there  exists 


at  least  one  row-vector  p  such  that 

NUMSTATES(Pj) 

pTI  =  p,  where  V/  pj  ^  0  and  L  Pi  =  1  (4) 

7=1 

where  p  is  a  stable  long  term  distribution  over  the  PFSA 
states.  If  n  is  irreducible,  then  p  is  unique.  Otherwise,  there 
may  exist  more  than  one  possible  solution  to  Eq.  10,  one  for 
each  eigenvector  corresponding  to  unity  eigenvalue.  However, 
if  the  initial  state  is  specified  (as  it  is  in  this  paper),  then 
p  is  always  unique.  Several  efficient  algorithms  have  been 
reported  in  the  literature  [14][15][16]  for  computation  of  p. 


Key  definitions  and  results  from  Measure  theory  that  are 
used  here  are  recalled. 

Definition  2.4:  (a-Algebra)  A  collection  9)1  of  subsets  of 
a  non-empty  set  X  is  said  to  be  a  o-algebra  [17]  in  X  if))l  has 
the  following  properties: 


1)  X  e  9)1 

2)  If  A  e  9)1,  then  Ac  e  9)1  where  Ac  is  the  complement  of  A 
relative  to  X,  i.e.,  Ac  =  X\  A 

3)  If  A  =  U”i  A„  and  if  An  e  9)1  for  n  e  IN,  then  A  e  9)1. 
Theorem  2.1:  If  (F  is  any  collection  of  subsets  of  X,  there 

exists  a  smallest  o-algebra  9)1*  in  X  such  that  (F  Q  9)1*. 

Proof:  See  Theorem  1.10  in  [17],  7] 

Definition  2.5:  (Measure)  A  finite  (non-negative)  mea¬ 
sure  is  a  countably  additive  function  y,  defined  on  a  o-algebra 
9)1,  whose  range  is  [0,  f(]  for  some  KeR  Countable  additivity 
means  that  if{Aj}  is  a  disjoint  countable  collection  of  members 
of  9)1,  then 


fi  UAi  =X,P(Ai)  (5) 

V  i= 1  i-l 

Theorem  2.2:  If  y  is  a  (non-negative)  measure  on  a  o- 
algebra  9)1,  then 


1)  p(0)  =  0 

2)  (Monotonicity)  AcB  ==>  y  (A)  <  y(B)  if  A,  B  e  9)1. 

Proof:  See  Theorem  1.19  in  [17],  □ 

Definition  2.6:  A  probability  measure  on  a  non-empty  set 
with  a  specified  o-algebra  9)1  is  a  finite  non-negative  measure 
on  9)1.  Although  not  required  by  the  theory,  a  probability 
measure  is  defined  to  have  the  unit  interval  [0, 1]  as  its  range. 

Definition  2.7:  A  probability  measure  space  is  a  triple 
(X,9)l,p)  where  X  is  the  underlying  set,  9)1  is  the  o-algebra 
in  X  and  p  is  a  finite  non-negative  measure  on  9)1. 


3.  Properties  of  Probabilistic  Finite  State 
Automata 

For  any  r  e  E*,  the  language  tE“  has  an  important 
physical  interpretation  pertaining  to  systems  modeled  as 
probabilistic  language  generators  (See  Fig.[5]i.  A  string  x  e  E* 
can  be  interpreted  as  a  symbol  sequence  that  has  been  al¬ 
ready  generated,  and  any  string  in  Ew  qualifies  as  a  possible 
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future  evolution.  Thus,  the  language  xE"  is  conceptually 
associated  with  the  current  dynamical  state  of  the  modeled 
system. 


Possible 

Future 


Fig.  5.  Interpretation  of  the  language  xE"  pertaining  to 
dynamical  evolution  of  a  language  generator 


Definition  3.1:  Given  an  alphabet  E,  the  set  23  £  =  2E  E" 
is  defined  to  be  the  a-algebra  generated  by  the  set  |l  :  L  = 

xE“  where  x  £  E*j,  i.e.,  the  smallest  a-algebra  on  the  set  TP, 

which  contains  the  set  j  L  :  L  =  xE where  x  £  E*J. 

Remark  3.1:  Cardinality  of 'Hz  is  Kj  because  both  2E*  and 
E'"  have  cardinality  N,. 

The  following  relations  in  the  probability  measure  space 
(E“,  23E/p)  are  consequences  of  Definition  13. II 
.  p(E“)  =  p(eE“)  =  1 

.  lx,  u  e  T* ,  xuEw  Q  xE“  and  hence  p(xuTa)  g  p(E“) 
Notation  3.1:  For  brevity,  the  probability  p(xE11')  is  de¬ 
noted  as  p(x)  Vx  €  E*  in  the  sequel. 


Next  the  notion  of  probabilistic  Nerode  equivalence  A/p 
is  introduced  on  E*  for  representing  the  measure  space 
(TP ,  23 v,  p)  in  the  form  of  a  PFSA.  In  this  context,  the  fol¬ 
lowing  logical  formulae  are  introduced. 

Definition  3.2:  For  x,  y  £  E*, 


Ui(x,  y)  =  (p(x)  =  0/\  P(y)  =  0) 
v2(x,  y)  =  (p(x)  10  /\  P(y)  i  o)  f\ 


(6a) 


(v„ez*(£^  =  M 
\  PM  P (y)  J. 

Nerode 


(6b) 

\  pw  pw  // 

Theorem  3.1:  (Probabilistic  Nerode  Equivalence) 

Given  an  alphabet  E,  every  measure  space  (E“,  23E,  p)  induces 
a  right-invariant  equivalence  relation  A/p  on  E*  defined  as: 


lx,  y  e  E*,  ( xNpy  Ui(x,  y)  \f  U2(x,  y) j  (7) 

Proof:  Reflexivity  and  symmetry  properties  of  the 

relation  Np  follow  from  Definition  13.21  Let  x,y,z  £  E*  be 
distinct  and  arbitrary  strings  such  that  xNpy  and  yA/pz. 
Then,  transitivity  property  of  A/p  follows  from  Eq.  fT}  and 
Definition  13.21  Hence,  Np  is  an  equivalence  relation. 

To  establish  right-invariance  [13]  of  A/p,  it  suffices  to  show 
that 


lx,  y  e  E*,  xA/py  =>  lu  £  E 


' ,  (xuNpyiij 


(8) 


Let  x,y,u  be  arbitrary  strings  in  E*  such  that  xNpy.  If  p(x)  = 
0,  p(y)  =  0  from  Eq.  0.  Then,  it  follows  from  the  monotonicity 
property  of  the  measure  (See  Theorem  12. 2t  that  p(xu)  =  0, 
which  implies  the  truth  of  Ui  (xu,yu)  and  hence  the  truth  of 
xuNpyu.  If  p(x)  1  0,  then  (xA/py)  A  (pM  1  0)  implies  p(y)  1  0. 
Hence, 

P  (xut)  =  p  (xmt)  p(x) 
p  (xu)  p(x)  p  (xu) 

If  p(x)  =  p(y),  then  xNpy  implies  p (xu)  =  p (yu)  and  also  Vx  £ 
E*(p(xkt)  =  p(ywx)).  Similarly,  if  p(x)  1  p(y),  then  xNpy  implies 
p (xu)  1  p (yu)  and  also  Vx  £  E*(p(xux)  1  p(yux)).  Hence,  Vx  £ 

E*((p(xu)  =  p(yw))  (P (xut)  =  p(yux))j.  □ 

Definition  3.3:  (Perfect  Encoding)  Given  an  alphabet 
E,  PFSA  Pi  =  (Q,  E,  5,  qt,  ft)  is  defined  to  be  a  perfect  encoding 
of  the  measure  space  (E“,  23E,  p)  if  Vx  £  E+  and  x  =  a\a2  •  ■  ■  or. 


P(x)  =  n(qi,o1)Y\n(.h*(qi,o1---ak),ok+1)  (10) 

k= 1 

Remark  3.2:  The  implications  of  Definition  13.31  are  as 
follows:  The  encoding  introduced  is  perfect  in  the  sense  that 
the  measure  p  can  be  reconstructed  without  error  from  the 
specification  of  Pi. 

Theorem  3.2:  A  PFSA  is  a  perfect  encoding  if  and  only  if 
the  corresponding  probabilistic  Nerode  equivalence  A/p  is  of 
finite  index. 

Proof:  (Left  to  Right:)  Let  Q  be  the  finite  set  of 
equivalence  classes  of  the  relation  A/p  of  the  PFSA  P,  = 
(Q,L,5,qi,n)  that  is  constructed  as  follows: 

1)  Since  Np  is  an  equivalence  relation  on  E*,  there  exists 
a  unique  q,  £  Q  such  that  £  £  q,.  The  initial  state  of  P,  is 
set  to  qt. 

2)  If  x  £  qj  and  xo  £  %  then  bfqj.o)  =  q* 

3)  n(qj,  o)  =  where  x  £  qr 

First  we  verify  that  the  steps  2  and  3  are  consistent  in  the 
sense  that  5  and  ft  are  well-defined. 

Probabilistic  Nerode  equivalence  (See  Theorem  13. Ill  im¬ 
plies  that  if  x,  y  £  E*,  then  (  (x  £  q^j  A  (xo  £  qf)  A  (y  £  q^j  )  ==> 
{yo  £  qic).  Therefore,  the  constructed  5  is  well-defined.  Sim¬ 
ilarly,  since  (x,y  £  qj)  =>  (p(x)  =  p(y))  A  (p(xn)  =  p(yu)),  the 
constructed  ft  is  also  well-defined.  Therefore,  the  steps  2  and 
3  are  consistent.  For  x  =  01CT2  ■  •  ■  or  £  E+,  it  follows  that 

R 

p(x)  =  p(ui)  Y\  ' 


P(oi  ■■■Or) 


__  p(ffl  ■■■Or-l) 

R- 1 

=  h(qi,  o1)  ]^[  n(6*(qit  ax  ■  ■  ■  of),  or+1) 


Hence,  the  criterion  for  perfect  encoding  (See  De(initionl3.3t 
is  satisfied. 

(Right  to  Left:)  Let  the  PFSA  P,  =  (Q,  E,  5,  qi,  ft)  be  a  perfect 
encoding;  and  let  the  probabilistic  Nerode  equivalence  A/p  be 
of  infinite  index.  Then,  there  exists  a  set  of  strings  P/P  c  E*, 
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having  the  same  cardinality  as  £*,  such  that  each  element 
of  belongs  to  a  distinct  A/p -equivalence  class.  That  is, 
Vlij,lik  £  such  that  j  V  k,  we  have  hjjy^hk.  Since  p (hj)  = 
Vihk)  =  0  implies  hjNphk,  there  can  exist  at  most  one  element 
h0  e  Jif  such  that  p(/i0)  =  0.  That  is,  p (hj)  4  0  Vhj  e  M1  -  {h0}. 

For  the  PFSA  P,  =  ( Q ,  £,  b,  q„  ft),  where  Q  is  the  finite  set  of 
states,  there  exists  qe  £  Q  and  hj,  hk  £  JF  such  that  b*(qi,hj)  = 
b*(quhk)  =  qe.  Let  t  £  £+  and  t  =  o\02---or.  Since  P,  ia  a 
perfect  encoding,  it  follows  from  Definition  13.31  that 

r- 1 

P(V)  =  p(hj)n(q(,a1)Y\n(5*(q{,a1---am),am+1) 

m= 1 

r— 1 

P(fycT)  =  VQlk)fi(.qe,a\)  n(d*(qe,Oi  ■  ■■om),am+i) 

m= 1 


Now,  it  follows  that 


V  0  A  p (lik)  4  0 


P(V)  =  V(hkT))\ 
P (hj)  p(hk)  ' 


V2(hj,hk)  =>  hjNphk 


which  contradicts  the  initial  assertion  that  hj  /J/p  hk  Hi,,  lik  £ 
,/P.  This  completes  the  proof.  □ 

The  construction  in  the  first  part  of  Theoreml3.2lis  stated 
in  the  form  of  Algorithm  [T] 


Algorithm  1:  Construction  of  PFSA  from  the  probability 
measure  space  (£“,  23  y,  p) 
input  :  IT.'" ,  P)  such  that  Np  is  of  finite  index 

output:  Pi 

1  begin 

2  Let  Q  =  {(fy  :  /  e  J  g  nJ  be  the  set  of  equivalence  classes  of 
the  relation  Np; 

3  Set  the  initial  state  of  P;  as  q\  such  that  E  belongs  to  the 
equivalence  class  qi\  If  x  e  qj  and  xa  e  qk,  then  set 
b(qj,a)  =  qk; 

4  o)  =  ^  where  xeqj; 

5  end 


Corollary  3.1:  (to  Theorem  13.21  A  PFSA  P,  = 

(Q,  L,  5,  qi,  ft)  induces  a  probability  measure  p  on  the  o-algebra 
SE  and  the  corresponding  probabilistic  Nerode  equivalence  is 
of  finite  index. 

Proof:  Let  a  probability  measure  p  be  constructed  on 
the  a-algebra  23 E  as  follows: 

Vt  £  £+,  | p(x)  =  n(qt,  o2)  J^[  fi(6* (<?,-, ui  •  •  ■  ok),  ok+i) j 


It  follows  from  Definition  13.31  that  Pi  perfectly  encodes  the 
measure  p  and  Theorem  13.21  implies  that  the  corresponding 
Np  is  of  finite  index.  □ 

On  account  of  Corollary  13. 11  we  can  map  any  given  PFSA 
to  a  measure  space  (£",23e  ,p). 


Definition  3.4:  Let  be  the  space  of  all  probability  mea¬ 
sures  on  23e  and  si  be  the  space  of  all  possible  PFSA  Pi  = 
(Q,  L,  6,  qif  ft). 

•  The  map  H  \  si  PP  is  defined  as  H(P,)  =  p  such  that 

l  r_1  \ 

Vt  £  £+,  (p(t)  =  f%,  ui)  J^[  n(b*(qit  ak  ■  ■  ■  ok),  ak+ 1)  j 

'  Jc=l 

where  x  =  aka2  ■  ■  ■  or  (11) 

•  The  map  H_i  :  9*  — >  si  is  defined  as: 

{Pi  given  by  Algo.  [7]  if  Np  is  of  finite  index 
Undefined  otherwise 

(12) 

Lemma  3.1:  Pi  is  a  perfect  encoding  for  H(P,). 

Proof:  The  proof  follows  from  Definition  13.31  and  Defi¬ 
nition  13.41  □ 

Next  we  show  that,  similar  to  classical  finite  state  ma¬ 
chines,  an  arbitrary  PFSA  can  be  uniquely  minimized.  How¬ 
ever,  the  sense  in  which  the  minimization  is  achieved  is 
somewhat  different.  To  this  end,  we  introduce  the  notion  of 
reachable  states  in  a  PFSA  and  define  isomorphism  of  two 
PFSA. 

Definition  3.5:  (Reachable  States)  Given  a  PFSA  Pi  = 

(Q,  L,  5,  qt,  h),  the  set  of  reachable  states  Rch(P,)  QQis  defined 
as: 

q  £  Rch(P,)  =>  3t  =  cq  ■  ■  •  Or  £  £*  such  that 

R- 1 

(b*(qi,  t)  =  q)  /\  ( n(qi ,  ak)  ]^[  ft(6*(qi,  o1  ■  ■  ■  ar), or+1)  >  0) 

r=l 

Remark  3.3:  The  strict  positivity  condition  in  Defini- 
tion  \3.5\ensures  that  every  state  in  the  set  of  reachable  states 
can  actually  be  attained  with  a  strictly  non-zero  probability. 
In  other  words,  for  every  state  qj  £  RCH(P;),  there  exists  at  least 
one  string  w,  initiating  from  qi  and  eventually  terminating  on 
state  qj,  such  that  the  generation  probability  of  co  is  strictly 
positive. 

Definition  3.6:  (Isomorphism:)  Two  PFSA  P,  = 

(Q,L,b,qi,h)  and  PJ,  =  (Q’  ,L,5’  ,q'i:,n')  are  defined 

to  be  isomorphic  if  there  exists  a  bijective  map 
r|  :  RCH(P,)  — >  RCH(P',)  such  that 

n(qjro)  V  0  =>  yn'(r\(qj),o)  =  n(qj,a) j  /\ 

(b'(i](qj),0k)  =  r\(b(qj,0k))j 

Remark  3.4:  The  notion  of  isomorphism  stated  in  Defi¬ 
nition  13.61  generalizes  graph  isomorphism  to  PFSA  by  con¬ 
sidering  only  the  states  that  can  be  reached  with  non-zero 
probability  and  transitions  that  have  a  non-zero  probability 
of  occurrence. 

Theorem  3.3:  (Minimization  of  PFSA:)  For  a  PFSA  Pi  = 

(Q,  L,  5,  qt,  h),  H_j  o  H(P;)  is  the  unique  minimal  realization  of 
Pi  in  the  sense  that  the  following  conditions  are  satisfied: 
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1)  The  PFSA  H_i  o  H(P,j  perfectly  encodes  the  probability 
measure  H(P,j. 

2)  For  a  PFSA  Pj,  that  perfectly  encodes  H(P,),  the  inequality 
Card(Rch(H_i  o  H(P,)))  ^  Card(Rch(P',))  holds. 

3)  The  equality,  Card(Rch(H_,  o  H(P,)))  =  CARD(RCH(P',)), 
implies  isomorphism  of  Pi  and  Pj,  in  the  sense  of  Defini¬ 
tion  13.61 

Proof: 

1)  The  proof  follows  from  the  construction  in  Theorem  13.21 

2)  Let  Pj,  =  (Q' ,  E, 5',  q,>,n')  be  an  arbitrary  PFSA  that 

perfectly  encodes  the  probability  measure  H(P;).  Let  us 
construct  a  PFSA  Pt,  =  (Q'  U  \qd),  I,  <5+,  where  qd  is 

a  new  state  not  in  Q',  as  follows: 


Vp' £Q',oeE, 

qd  if  tt’WyOk)  =  0 

6'  (q'j,  of)  otherwise 

Va  £  I,  5*(qdr  ok)  =  qd 
Vq'i  e  Q',  Vcr  £  E,  n\q'jrok)  =  n'(q'jrOk) 


(13a) 

(13b) 

(13c) 


It  is  seen  that  Pj,  perfectly  encodes  H(P,)  as  well,  which 
follows  from  Definition  13.31  and  Eq.  d  13cl>.  It  is  claimed 
that 


Card(Rch(P+))  =  Card(Rch(P',))  (14) 


based  on  the  following  rationale. 

Let  q'  e  RCH(Pj,).  Following  Definition  13.51  there  ex¬ 
ists  a  string  t  £  E*  such  that  b'*(qi',x)  =  q'.  and 
n’(qd,  Oi)  n't,'  ti' {&* (qi> , 0\  ■  ■  ■  or),  or+i)  >  0.  It  follows  from 
Eq.  d!3cl>  that  7T+(p,v,(j,)  n'ti'  ftf(bf*{qd,o^  ■  •  ■  or),  Ur+i )  >  0 
and  hence  we  conclude  using  Eq.  d!3ab  that  bi*(qd,x)  = 
qj  =£  qd  which  then  implies  that  q'.  £  RCH(Pj,).  Hence 
we  have  CARD(RCH(Pj,))  5S  CARD(RCH(Pl‘,)).By  a  similar 
argument,  we  have  Card(Rch(P1‘,))  ^  CARD(RCH(P',)) 
and  hence  Card(Rch(P1;))  =  Card(RCH(P',)). 

Next,  we  claim 


Vx,y  £  E*((<5+  (qd, x)  =  b\qr,y))  = =>  xAfopyy 


(15) 


based  on  the  following  rationale. 

Let  x,  y  £  E*  s.t.  (bf(qd,x)  =  bf(qd,  y)).  It  follows  from  Eqs. 
d!3ab  d!3bb  and  d  13cl>  that 


I  (H (Pi)(x)  =  0  A  H(P,j(y)  =  0)  if  bf(qd,x)  =  qd 
|  (H(P,j(x)  A  0  A  H(P,j(y)  A  oj  otherwise 


Now,  if  (H(P,)(x)  =  0  A  H(P;)(y)  =  o),  then  it  follows  from 
Eq.  ([6b)  that  xNuipq  y.  On  the  other  hand,  if  (H(P,j(x)  a 
0  A  H(P,)(y)  A  0),  then  Eq.  (l6at  yields: 


Vu  —  0\  •  •  •  (Tft  €  Tj 


H  (Pi)(xu) 
’  H(Pf)W 


H  (P,)(yu) 
H(P,)(y) 


R- 1 

=  nf(qd,0\)  ]^[  ft+(6+(^,  Ui  ■  •  ■  or),  or+1)  =>  xJVH(Pi)  y 

r=l 

We  define  a  map  C  :  Rch(H_!  o  H(P,))  — >  RCH(Pl‘/)  as 
follows:  Let  q *  £  Rch(H_1  o  H(P,))  and  let  <S(q*)  be  the 
equivalence  class  of  the  relation  Afepy  represented  by 
q*.  Let  x  =  0\  ■  ■  ■  Or  £  S(q#). 


H(P;)(x)  >  0  (See  Definition  [3131 

R-l 

=>  7t+(^v,CTi)  ■■■Or),0r+l)  >  0 

r- 1 

(Since  Pf,  perfectly  encodes  HfP,)) 

=>  5f*(qi',x)  £  RCH(P+) 

Let  C(<7#)  =  <5+*  ((/;/,  x).  Note  that  C(<?#)  depends  on  the 
choice  of  x.  Let  q\,q*2  £  Rch(H_!  o  H(P,j)  such  that  L,(q*)  = 
C(t/2).  If  x, ,  x2  are  the  corresponding  strings  chosen  to 
define  C(<?f),  C(<?2),  we  have  b^iqi^x^  =  6+*(p,/,x2)  which 
implies  x1A4I(i>j)X2,  i.e.,  q\  =  q *■  Hence  we  conclude  C  is 
injective  which,  in  turn,  implies 

Card(Rch(H_,  o  H(P,)))  ^  Card(Rch(PJ,))  (17) 

Finally,  from  Eqs.  d  14b  and  d  17b.  it  follows  that 

Card(Rch(H_i  o  H(P,)))  ^  Card(Rch(P',))  (18) 

3)  Let  Pj,  =  (Q',  E, b' ,  q,',h')  be  an  arbitrary  PFSA  that 
perfectly  encodes  H(P,j  such  that 


Card(Rch(H_i  o  H(P;)))  =  Card(Rch(P',))  (Cl) 

Let  the  PFSA  H_,  o  H(P;)  be  denoted  as  (Q#,  E,  5*,  q n*). 
Let  S{qp  denote  the  equivalence  class  of  ACHpy  that  q* 
represents.  We  define  a  map  cp  :  Rch(H_!  o  H(P,j)  — > 
2RCH(pi')  as  follows: 


<#»(<?*)  =  {<?;  e  Q'\3x  £  S{q]b)  s.t.  5'*(qi',x)  =  q '}  (19) 

We  claim 

V<7*,  4  e  Q*((4  *  4)  =»  n  &&  =  0))  (C2) 

Let  q’(  £  <p(q#)  n  <p(4)-  Hence  there  exists  Xj  £  $(q*),Xk  £ 
<o(q*k)  such  that 


q't  =  b'*{qv,Xj)  =  5’*(qi',xk) 


(20) 


that 

;  ™  \  H(P;)(Xy)  *  H (P,-)(xt) 

But,  Pj,  perfectly  encodes  H(P,j  implying 

/H(PI)(x,u)  H(P,)(x,u) 

1  H(P;)(x;)  H(P,)(xt) 


Vif  —  0\  •  •  •  On  £  If 


R- 1  \ 

=  fc'(q'e,Oi)  ]^[  h'(b'*(ql',Oi  ■  ■  ■  ar),ar+i)| 

r=  1  ' 


(21) 
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which  contradicts  Eq.  <f2~0. 
Next  we  claim  that 


Vq*  £  RCH(H_!  o  H(P;)),  CARD(tp(q*))  =  1 


Let  xi,x2  £  ^(q#)  such  that 


6'*  (<?/',  xi)  =  q'j 

S'*  (<?;-,  x2)  =  q'k 


with  A  q’k 


(C3) 


(22) 


Therefore, 


Card ((p(q*))  >  1 

=>  ^  Card (cp(ql))  >  Card(Rch(H_!  o  H(P,))) 

?*eRCH(H_1oM(P;)) 

=>  CARD(RCH(P',))  >  CARD(RCH(H_!  O  H(Pj))) 

which  contradicts  Cl  thus  proving  C3. 

On  account  of  C2  and  C3,  let  us  define  a  bijective  map 
45  :  RCH(H_!  o  H(P,))  -»  RCH(P',)  as  §(if  )  =  6'*(qi>,x)/X  £ 
Then, 


4ok  £  L,Vq*  £  RCH(H_!  o  H(P,))),x  e  £(qfi, 

c \>(5*(q*,ok))  =  5'*(qi',xok) 

=  5'(5'*(qi*,ok),ok)  =  6'(§(q%ok) 


(23) 


(24) 


which  implies  that  TT_i  o  H(P;)  and  P',  are  isomorphic  in 
the  sense  of  Definition  13.61  This  completes  the  proof. 

□ 

Theorem  3.4:  For  a  PFSA  Pi  =  (£' ,  E,  5,  Ei,  ft),  the  function 
ft  :  Q  x  E  — >  Q  can  be  extended  to  ft  :  QxE*  — >  Q  as: 

Vq,  £  Q,t  e  E\a  e  E,  j  “  1  (25) 

\  n(qj,OT)  =  n(qj,o)n(b{qj,o),T) 

Proof:  Let  p  =  H(P,).  We  note  that  that  P,  perfectly 

encodes  p  (See  Lemma [3. Ill .  It  follows  from  Theoreml3.2lthat 

e  QMqp  E)  =  =  1  where  6*  (<?,■,  *)  =  qf 

Similarly,  for  a  string  ox  initiating  from  state  qj,  where  a  £ 
E,  x  e  E*,  we  have 


n(qjroi)  = 


p (xot)  _  p(xcr)  p(xctt) 
P(x)  P(x)  p  (xo) 


Proof:  Let  H_i(p)  =  P;  =  (Q,E,  5,qi,  ft).  We  note  P, 
perfectly  encodes  p  (See  Lemmal37fl>.  Let  H(P,)  =  p'.  We  claim 

Vx  e  E*,p(x)  =  p'(x) 

The  result  is  immediate  for  |x|  =  0,  i.e.,  x  =  e.  For  |x|  >  1,  we 
proceed  by  the  method  of  induction.  For  |x|  =  1,  we  note 

Vcr  £  E,p \o)  =  n(qi,o)  =  p(cr)  (Perfect  Encoding) 

Next  let  us  assume  that  Vx  £  E*,  s.t.  |x|  =  r  £  N,  p'(x)  =  p(x). 
Since  Vx  e  E*  with  |x|  =  r  £  N,  it  follows  that 

p'(xcr)  =  p '(x)n(qj,o)  where  5*(qi,x)  =  qj 
=  P(x)n(qjro)  =  p(xct) 

This  completes  the  proof.  □ 

4.  Metrization  of  the  Space  of  Probability 
Measures  on  23e 

Metrization  of  S?  is  important  for  differentiating  physical 
processes  modeled  as  dynamical  systems  evolving  probabilis¬ 
tically  on  discrete  state  spaces  of  finite  cardinality.  In  this 
section,  we  introduce  two  metric  families,  each  of  which 
captures  a  different  aspect  of  such  dynamical  behavior  and 
can  be  combined  to  form  physically  meaningful  and  useful 
metrics  for  system  analysis  and  design. 

Definition  4.1:  Given  two  probability  measures  px,p2  on 
the  o-algebra  SE  and  a  parameter  s  £  [1, 00],  the  function  ds  : 
^x^^[0,l]  is  defined  as  follows: 


/  H 

ds(Pi,P2)  =  SUP(  X 
;=1 


xeL* 


Pl  (xoj)  p2(xu;)| 


PlM 

doo(Pi,p2)  =  sup  max 


p2(x)  I 
Pl  (X0j) 


l/s 


Vs€[l,oo)  (28a) 
|  (2gb) 


(26) 


We  note  that  =  h(qj,o).  Also,  6*(qi,x)  =  qj  implies 
b{qj,o)  =  5*(qi,xo).  Therefore,  =  h(b(qj,o),x)  and  hence 

n(qj,oi)  =  n{qj,o)n(b{qj,a),  x) 

This  completes  the  proof.  □ 

Theorem  3.5:  For  a  measure  space  (E“,  23E ,  p), 

H  o  H_i(p)  =  p  (27) 

i.e.,  H  o  H_i  is  the  identity  map  from  3?  onto  itself. 


xeZ*  atz.  ,  Pj(x)  P2(X)  I 

Theorem  4.1:  The  space  3P  of  all  probability  measures  on 
SE  is  ds-metrizable  for  s  £  [1, 00]. 

Proof:  Strict  positivity  and  symmetry  properties  of  a 
metric  follow  directly  from  Definition  14.11  Validity  of  the 
remaining  property  of  triangular  inequality  follows  by  ap¬ 
plication  of  Minkowski  inequality  [17].  □ 

Definition  4.2:  Let  M  be  a  right  invariant  equivalence 
relation  on  E*  with  the  ith  equivalence  class  of  M  be  de¬ 
noted  as  M ,  i  £  I,  where  I  is  an  arbitrary  index  set.  Let 
p  be  a  probability  measure  on  the  o-algebra  SE inducing 
the  probabilistic  Nerode  equivalence  A/p  on  E*  with  the  jtk 
equivalence  class  of  A/p  denoted  as  Nl,  j  £  J,  where  J  is 
an  index  set  distinct  from  I.  Then,  the  map  Q M  \  — > 

[0,1]Card(J)  x  [0,1]°™  is  defined  as 


Oai(P) 


=  Yj  P(x) 
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Definition  4.3:  Let  px,  p2  be  two  probability  measures  on 
the  o-algebra  23j;.  Then,  the  function  dF  :  x  3A  — >  [0, 1]  is 

defined  as  follows: 

MPuh)  =  ||Q^p2(Pi)  -  nA/Pi(P2)||f  (29) 

where  ||@||f  =  ^/Trace[&H&]  is  the  Frobenius  norm  of  the 
operator  0,  and  &H  is  the  Hermitian  of  0. 

Definition  14.31  implies  that  if  I  and  fj  are  the  index  sets 
corresponding  to  ACPl  and  ;Vp2  respectively,  then  (p2)  £ 
[0,1]CARD(J)  x  [0,1]CARD(J)  and  Ojvp  (pj)  e  [0, 1]CAR’D(J)  x 

[Q,  1jCARD(J)_ 

Theorem,  4.2:  The  function  dp  is  a  pseudometric  on  the 
space  3?  of  probability  measures. 

Proof:  The  Frobenius  norm  on  a  probability  space 
satisfies  the  metric  properties  except  strict  positivity  because 
of  the  almost  sure  property  of  a  probability  measure.  □ 

Theorem  4.3:  For  4 a  e  [0, 1)  and  Vs  e  [1,  oo],  the  parame- 
tized  function  fia  s  =  adF  +  (1  -  a)ds  is  a  metric  on  3P. 

Proof:  Following  Theorems  14.11  and  14.21  ds  is  a  metric 
for  s  6  [1,  oo]  and  dp  is  a  pseudometric  on  3s.  Non-negativity, 
finiteness,  symmetry  and  sub-additivity  of  p  follow  from 
the  respective  properties  of  dF  and  ds.  Strict  positivity  of 
on  a  £  [0, 1)  is  established  below. 

(Pi '  P2)  =  0  =>  (1  —  a)ds(p1,p2)  =  0  =>  Pj  =  p2  (30) 

□ 


Remark  4.2:  Corollary  14.11  can  be  physically  interpreted 
to  imply  that  the  metric  family  va/S  does  not  diffrentiate  be¬ 
tween  different  realizations  of  the  same  probability  measure. 
Thus  when  comparing  two  probabilistic  finite  state  machines, 
we  need  not  concern  ourselves  with  whether  the  machines 
are  represented  in  their  minimal  realizations;  the  distance 
between  two  non-minimal  realizations  of  the  same  PFSA  is 
always  zero.  However  this  implies  that  vaf,  only  qualifies  as 
a  pseudo-metric  on  si. 

A.  Explicit  Computation  of  the  Pseudometric  v  for 
PFSA 

The  pseudometric  v„^  is  computed  explicitly  for  pairs  of 
PFSA  over  the  same  alphabet.  Before  proceeding  to  the 
general  case,  vafi  is  computed  for  the  special  case,  where 
the  pair  of  PFSA  have  identical  state  sets,  initial  states  and 
transition  maps. 

Lemma  4.1:  Given  two  PFSA  P'  =  (Q,  L,b,  qi,  ft1),  P2  = 
(Q,L,5,qi,n2),  and  o  £  E,  the  steps  for  computation  of 
Vo Apt'p?)  are: 

Set  A(qf)  =  n\qj,o)  -  n2{qjro) 

Then,  v0 u(P*,P-)  =  mmc||A(^)||s 

Proof:  Let  denote  a  |£|-dimensional  vector¬ 

valued  function,  where  a  £  E.  For  proof  of  the  lemma,  it 
suffices  to  show  that  the  following  relation  holds: 


Remark  4.1:  If  two  physical  processes  are  modeled  as 
discrete-event  dynamical  systems,  then  the  respective  prob¬ 
abilistic  language  generators  can  be  associated  with  proba¬ 
bility  measures  p:  and  p2.  The  metric  is(Pi,P2)  is  related  to  the 
production  of  single  symbols  as  arbitrary  strings  and  hence 
captures  the  difference  in  short  term  dynamic  evolution.  In 
contrast,  the  pseudometric  dF  is  related  to  generation  of  all 
possible  strings  and  therefore  captures  the  difference  in  long 
term  behavior  of  the  physical  processes.  The  metric  pa  s  thus 
captures  the  both  short-term  and  long-term  behavior  with 
respective  relative  weights  ofl-a  and  a. 

Definition  4.4:  The  metric  on  3?  for  a  £  [0, 1),  s  £ 
[l,oo],  induces  a  function  va ?  on  si  x  si  as  follows: 

VP„  p;,  £  ^,vaAPi,Pp)  =  ^S(H(P,),H(P',))  (31) 

Corollary  4.1:  (to  Theorem  1 4.3)  The  function  vafi  in 
Definition  14.41  for  a  £  [0,1)  and  s  £  [1, 00]  is  a  pseudometric 
on  si.  Specifically,  the  following  condition  holds: 

va,s(Pi,H-1  o  H(P,))  =  0  (32) 

Proof:  Following  Theorem  13.51 

va,s(P„  H_!  o  H(P;))  =  ^(H  (Pi),  Bol-jo  H(P,)) 

=  pJH (P,),(H  o  H_x)  o  H(P,))  =  pJMPMiiP,))  =  0 

□ 


supd!W,5(W\.max||  || 

Since  Pj  perfectly  encodes  H(Pj),  it  follows  that 
(  Vx,y  £  L*,5(qi,x)  =  b(qt,yf)  implies 

H(P})(xok)  _lt  H (PjXyo) 

H (Pj)(x)  U  {qi'Gk’  H(pi)(y) 

where  b(qi,x )  =  qj.  Similar  argument  holds  for  H(P^).  Hence, 
it  follows  that  for  computing  v0iS(P2,P2),  only  one  string  needs 
to  be  considered  for  each  state  qj  £  Q.  That  is, 

H  (P})(xok)  H  (P2)(xok) 


v0AP]rP')=  max 

x:6(flipc)=qj 


H(Pj)(x)  H  (P2)(x) 


=  rCkj  ~  n2{q>'ak)l  = 

□ 

Lemma  4.2:  Let  pi,  p2  be  the  stable  probability  distribu¬ 
tions  for  PFSA  Pj  =  (Q,  E,  b,q:,  ft1)  and  P2  =  (Q,L,b,qi,n2) 
respectively.  Then, 

limva,s(P},P2)  =  d2(pi,p2) 

Proof:  Since  Pj  and  Pf  have  the  same  initial  state  and 
state  transition  maps, 

0  if  j  4  k 


N’ 

H(Ph 


n«i 


■k 

H  (Jf) 


N 1  ,  =  Nk  ,  otherwise 

H(P1)  H(Ph 
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where  A ,  and  )V|p(p2)  are  the  f‘  and  kth  equivalence  classes 
(i.e.,  states  q,  and  qf)  for  P],  P2,  respectively.  The  result 
follows  from  Definition  14. 3 1  and  C orol  1  ary  14.41  an d  noting  that 


Q 


WH(p>)) 


E  PM  =  pilj 

x-i  (qiJ)=qj 

0 


if  j  =  k 


otherwise 


□ 

Theorem  4.4:  Given  two  PFSA  P]  =  (Q,  E,  b,  q,,  fp )  and 
P '?  =  (Q,  E,  b,  qi,  ft2),  the  pseudometric  va,s(Pl,P?)  can  be  com¬ 
puted  explicitly  for  a  £  [0, 1)  and  s  £  [1,  oo]  as: 


vv(P},Pf)  =  alimv*,(Pj,P?)  +  (1  -  a)v0,s(P] , P?)  (33) 

a— »1 

Proof:  The  result  follows  from  Theorem  14.31  and  Corol¬ 
lary  14.41  n 

The  algorithm  for  computation  of  the  the  pseudometric  v 
is  presented  below. 


Algorithm  2:  Computation  of  v„,s(P„P',) 
input  :  P;,P',,s,a 
output:  VaAPi'P'i’) 

1  begin 

2  Compute  P12  =  (Q,  E,  6,  n  12)  =  Pi  ®  P', ; 

3  Compute  P21  =  (Q,  E,  6,  7t2i)  =  Pi,  <g>  P,-; 

4  for  j  =  1  to  CARD(Q)  do 

5  |  AO)  =  ||fti2(9y/ffit)  -  *2i(<7;,ffjt)||s; 

6  endfor 

7  v0,s  (Pir  P[, )  =  max q  A  (;) ; 

8  Compute  j3i2  ;  /*  State  Prob.  for  P22(Def.  12  ■  31)  */ 

9  Compute  j32i  ;  /*  State  Prob.  for  P2i(Def.  12 . 31)  */ 

10  Compute  d  =  ||gPi2  -  g»2i  ||2  ; 

11  Compute  VaAPuP-,)  =  ad  +  (  1  -  a)v0,s(P;,Pi,); 

12  end 


To  extend  the  approach  presented  in  Lemma  14.11  to  ar¬ 
bitrary  pairs  of  PFSA,  we  need  to  define  the  synchronous 
composition  of  a  pair  of  PFSA. 

Definition  4.5:  The  binary  operation  of  synchronous  com¬ 
position  of  PFSA,  denoted  as  ®  :  &  x  £?  — >  ■¥’,  is  defined  as 
follows: 


Let  l  Pi  = 

Gf  =  (Q',  E,  6',  <j'„  ft') 

Then,  P,  ®  Gy  =  (Q  x  Q',  E,  5®,  ((/;, ft®),  where 

(^b(qj,o),b'(q'.,,ofj,  ifb(qq,o)  and 

b®((qq,q'k,),a)  =  •  b'(q'.,,o)  are  defined. 

Undefined  otherwise 

(34) 

ft®((<7p<7it')'ff)  =  ft(<?po)  (35) 

Remark  4.3:  Synchronous  composition  for  PFSA  is  not 
commutative,  i.e.,  for  an  arbitrary  pair  Pi  and  Gy, 


Pi  <g>  G,v  V  Gy  <g>  P,- 


(36) 


Synchronous  composition  of  PFSA  is  associative,  i.e., 

VP’,,  4  e  (Pn  ®  Pi 2)  ®  4  =  P)i  ®  (P?2  ®  P|) 

Theorem  4.5:  For  a  pair  of  PFSA  Pi  and  Gy  over  the  same 
alphabet, 

H(Pj)  =  H(P,  ®  Gy)  (37) 

Proof:  Let  p  =  H(P,)  and  p'  =  H(P;  <g>  G,-).  It  suffices  to 
show  that 

Vx  G  E*,p(x)  =  p'(x)  (C4) 

For  |x|  =  0,  i.e.,  x  =  e,  the  result  is  immediate.  For  |x|  2:  1, 
we  use  the  method  of  induction.  Since  P,  perfectly  encodes 

H  (Pi), 

Va  £  E,  p(cr)  =  n(cji,  a) 

=  ^((qi,q'i,),a)  =  ^(a) 

Hence  C4  is  true  for  |x|  g  1. 

With  the  induction  hypothesis 

Vx  e  E*,  s.t,  |x|  =  r  e  N,  p(x)  =  p'(x)  (38) 

we  proceed  with  an  arbitrary  <7  £  E  to  yield 
p(xcr)  =  p (x)h(qj,  a)  where  b(qi,  x)  =  qq 
=  P '(x)n®((qq,q'j,),o)  where  =  (qpq'y) 

=  P  ’(xa) 

This  completes  the  proof.  □ 

Theorem  4.6:  Given  a  pair  of  PFSA  Pi,  PI,  and  an  arbi¬ 
trary  parameter  s  £  [1, 00],  Algorithm  \2}  computes  VaAPi/P'y) 
for  a  £  [0, 1),  s  £  [1, 00]. 

Proof:  By  Theorem  14.51  Va  £  [0, 1),  s  £  [1, 00], 

VaAPi'P'y)  =  vaAPi®P'yrPy®Pi )  (39) 

Since  P;  <g)  Pj,  and  Pj,  (g>  P,  have  the  same  state  sets,  initial 
states  and  transition  maps  (See  Definition  14. 5I>.  correctness 
of  Algorithm  [2]  follows  from  Lemmas  14.11  and  14.21  □ 

Example  4.1:  The  theoretical  results  of  Section  0  are  il¬ 
lustrated  with  a  numerical  example.  The  following  PFSA  are 
considered: 

pln  =  ({<h/ 92,93},  {0,1},  (40) 

pu  =  {0,1}, b2,qA,n2)  (41) 

as  shown  in  Fig.  \6\  and  Fig.  [7]  respectively,  and  Fig.  0 
illustrates  the  computed  compositions  P}n  <g)  P2A  (above)  and 
P]A  ®  ph  (below). 

Following  Algorithm  [2]  we  have 


||0.4  -  0.9 

0.6-O.lH, 

||0.2  -  0.9 

0.8-O.lH, 

||0.3  -  0.9 

0.7-O.lH, 

110.3-0.7 

0.7  -  0.3||, 

110.4-0.7 

0.6  -  0.3||, 

||0.2  -  0.7 

0.8-0.3H,  . 
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0/0.3 


Fig.  7.  PFSA  P2 


1/0.3 


Fig.  8.  P1  ®  P2  (above)  and  P2  <g>  P1  (below) 


As  an  illustration,  we  set  s  =  oo.  Hence, 

A„  =  [  0.5  0.7  0.6  0.4  0.3  0.5  f  (43) 
=>  VoAP^Pl)  =  ntax( A„)  =  0.7  (44) 

The  final  state  probabilities  are  computed  to  be 

fpi^pi  =  [0.15  0.07  0.09  0.2  0.22  0.27]  (45) 

Ppiqpi  =  [0.29  0.29  0.29  0.043  0.043  0.043]  (46) 

For  a  =  0.5,  we  have 

VosAplqi>P2u)  =  0.5d2(fppi®p2,  pP2®pi)  +  0.5  x  0.7 
=  0.4599 

The  pseudonorm  v0,oo (Pj^P^)  =  0.7  is  interpreted  as  fol¬ 
lows.  There  exists  a  string  x  e  E*  and  an  event  a  e  E 
such  that  probability  of  occurrence  of  a,  given  that  x  has 
already  occurred,  is  70%  more  in  one  system  compared  to  the 
other.  Also,  the  occurrence  probability  of  any  event,  given  an 
arbitrary  string  has  already  occurred,  is  different  by  no  more 
than  70%  for  the  two  systems.  The  composition  P^  ®P2U  shown 
in  the  upper  part  of  Fig.  [S]  is  an  encoding  of  the  measure 
H/Pj)  and  hence  is  a  non-minimal  realization  of  Pj,  while 


the  composition  P2a  ig>  P'qi  shown  in  the  lower  part  of  Fig.  [S] 
encodes  H(P2)  and  therefore  is  a  non-minimal  realization 
of  P2.  Although  the  structures  of  the  two  compositions  are 
identical  in  a  graph-theoretic  sense  (i.e.  there  is  a  graph 
isomorphism  between  the  compositions),  they  represent  very 
different  probability  distributions  on  SE. 


5.  Model  Order  Reduction  for  PFSA 


This  section  investigates  the  possibility  of  encoding  an 
arbitrary  probability  distribution  on  ®Eby  a  PFSA  with  a 
pre-specified  graph  structure.  As  expected,  such  encodings 
will  not  always  be  perfect.  However,  we  will  show  that  the 
error  can  be  rigorously  computed  and  hence  is  useful  for  very 
close  approximation  of  large  PFSA  models  by  smaller  models. 

Definition  5.1:  The  binary  operation  of  projective  compo¬ 
sition  ®  :  SP  x  SP  — »  SP  is  defined  as  follows: 


Let  < 


Pi  =  (Q,L,6,tfi,ft) 

G?  =  (Q',L,5',q'i„n’) 

G?  S>  Pi  =  (O'  x  Q,  E,  6®,  (q'^q,),  ft®) 


For  notational  simplicity  set  €  Q  and  Vq'k,  e  Q', 

Yj  (h(g‘')«) 

rf®  *((<)',,  wW 


Then,  G?®P,-  =  (Q,Lr6,qirn®)  s.t. 

a,  ?,-)**  ((<&,<&),*) 

ft®(^,<7)  =  -  * 


(47) 


eQ' 

Theorem  5.1:  For  PFSA  Pi,  Gj,  Lb  over  the  same  alphabet, 


1.  P;®(Gj®H*)  =  Pi®Hk 

2.  (Pi®Gj'j®Hk  #  P;®(Gy®Ht)  (Non-associative) 

3.  P;®G;-  ±  G;®P;  (Non-commut alive) 

Proof:  The  results  follow  from  Definition  15.  II  □ 


We  justify  the  nomenclature  "projective"  composition  in 
the  following  theorem. 

Theorem  5.2:  For  arbitrary  PFSA  Pi  and  G,-  over  the  same 
alphabet, 


(Gi>®Pi)®Pi  =  Gi’®Pi  (48a) 

Proof:  Let  P,  =  (Q,  E,  b,  c/„  ft).  Definition  15.11  implies 
that  (g,v®P;)  =  (Q,  E,  6,  qt,  ft*)  for  fp  computed  as  specified 
in  Eq.  <47D.  It  further  follows  from  Definition  15.11  that 
(Gi'®P,)®P;  =  (Q,  E, />,(/„  7t®),  i.e.  (G;/®P;)®P;  and  G;/®P; 
have  the  same  state  set,  initial  state  and  state  transition 
maps.  Thus,  it  suffices  to  show  that 


e  Q,  a  e  E,  fP(qj,  a)  =  ft®(ijy,  a)  (49) 


Considering  the  probabilistic  synchronous  composition 
(G'gjP;)  <8>P,  =  (Q  x  Q,  E,  6®,^,,*/,),  ft®)  (See  Definition [475), 


Vx  e  E*,  6®*((g„  q,),x)  =  (qj,  qj),  for  some  e  Q 
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It  follows  that,  for  qk  V  qk, 

Hqk,q,)=  Y  (h(G^P,-)(x))  =  0 


(50) 


x-.6®  ((<M,),x) 
=(«W/) 


Finally  we  conclude  Tqj  e  Q,  a  e  E, 

LneQHqk,qj)n^((qk,qj),a) 


E^Q  %*,<//) 


=  n*(cjj,cs)  (See  Definition  14. 5 


[GpJp 


■H(G,v)(xE/f)  =  n9(quqt) 


(54) 


where  Lj(  QL  such  that  a  e  Lj( 


b(qj,a)  =  qt  and  n®(qj,qf) 
is  the  j£lh  element  of  the  stochastic  state  transition  matrix 
If®  corresponding  to  the  PFSA  It  follows  from  Eq. 

<f54l>.  that 


(51) 

This  completes  the  proof.  □ 

Projective  composition  preserves  the  projected  distribution 
which  is  defined  next. 

Definition  5.2:  (Projected  Distribution: )  The  projected 
distribution  p  e  [0,  i]NuMSTATES(pi)  0fan  arbitrary  PFSA  G ?  with 
respect  to  a  given  PFSA  Pi  is  defined  by  the  map  [•]?,.  :  srf  — > 

[0/  1]NUMSTATES(Pj)  asf0U0ws: 

IGp]^  =  p  e  [o,  i]NuMSTATES(h)/ 

suchthat  if  A/7  is  the  fh  equivalence  class  (i.e.  the  jth  state)  of  Pi, 
then  ^  H(Gp)(x)  =  pj 

xeN) 

We  note  [Gp]p,.  is  a  probability  vector,  i.e., 


£  H(Gp)(xE;f)  =  JjGp 

< ijtQ 

=>[GpJp,  =y[G,v]p, 
e  q j^Q 


n®(qj,qt) 


n®(qj,qt) 


It  follows  that  [Gpjp;  satisfies  the  vector  equation 

[Gf.Jp,  =  [Gp]Pjn® 


(55) 


(56) 


We  note  that  |Gp  ®P,-]p,  is  the  stable  probability  distribution 
of  the  PFSA  Gp  <g>P;  and  hence,  we  have 


[Gp®Pfjp,  =  [Gp®Pijp,.n® 


(57) 


In  general,  a  stochastic  matrix  may  have  more  than  one 
eigenvector  corresponding  to  unity  eigenvalue  [18].  However, 
as  per  our  definition  of  PFSA  (See  Definition^. 2I>.  the  initial 
state  is  explicitly  specified.  It  follows  that  the  right  hand 
side  of  Eq.d53t  assumes  that  all  strings  begin  from  the  same 
state  qi  e  Q.  Hence  it  follows: 


Numstates(Pz) 

Y  =  E  h(g-')w = 1  (52) 

/= 1  i  xel.* 

Theorem,  5.3:  (Projected  Distribution  Invariance:) 

For  two  arbitrary  PFSA  Pi  and  G; .  over  the  same  alphabet, 

[GpJp,  =  [Gp®P1]pj 

Proof:  Let  P;  =  (Q,  E,  6,  (/;,  ft)  and  Gp  =  (Q',  S,  5',  <7,,,  ft'). 
It  follows  that  Gp®P;  =  (Q,Y.,b,qi,n®),  where  ft®  is  as 
computed  in  Definition  15.11  Using  the  same  notation  as  in 
Definition  15.11  we  have  Va  e  E, 

Y  H(Gp)(xo)  =  Y  $Wk"qj )n’Wk"a) 

x-.6*(qiJ)=qj  l'k'eQ’ 

'Z^eQ-  3(q'k,,qj)n’(q’k„o)] 


=  Y  Wwli) 


=  {  E 

i'eeQ' 


(53) 


Since  [Gp]p;  =  Lx-,6*^)^  h(gp)(x)  =  E $(q'k„qj),  it 
follows  that  Va  e  E, 

E x:6*([jijc)=qj  U(Gp  )(X0) 


[GpJp 


=  ft®(< IjrO) 


LEri*M=(j®(Gp)N  ns 

- 1 - =  229%,o) 

ff:6(?y,tT)=ijf  [GpJp,  ff:6(?;pr)=9f 


[GpJp;  =  [Gp  ®P;]pf 

(58) 

This  completes  the  proof. 

□ 

A.  Physical  Significance  of  Projected  Distribution  In¬ 
variance 

Given  a  symbolic  language  theoretic  PFSA  model  for  a 
physical  system  of  interest,  one  is  often  concerned  with 
only  certain  class  of  possible  future  evolutions.  For  exam¬ 
ple,  in  the  paradigm  of  deterministic  finite  state  automata 
(DFSA)  [8],  the  control  requirements  are  expressed  in  the 
form  of  a  specification  language  or  a  specification  automaton. 
In  that  setting,  it  is  critical  to  determine  which  state  of  the 
specification  automaton  the  system  is  currently  visiting.  In 
contrast,  for  a  PFSA,  the  issue  is  the  probability  of  certain 
class  of  future  evolutions.  For  example,  given  a  large  order 
model  of  a  physical  system,  it  might  be  necessary  to  work 
with  a  much  smaller  order  PFSA,  that  has  the  same  long¬ 
term  behavior  with  respect  to  a  specified  set  of  event  strings. 
Although  projective  composition  may  incur  a  representation 
error  in  general,  the  long-term  distribution  over  the  states  of 
the  projected  model  is  preserved  as  shown  in  Theorem  15.31 
The  idea  is  further  clarified  in  the  commutative  diagram  of 
Fig.  m 

Probabilistic  synchronous  composition  is  an  exact  repre¬ 
sentation  with  no  loss  of  statistical  information;  but  the 
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Fig.  9.  Commutative  Diagram  relating  probabilistic  composition, 
projective  composition  and  the  original  projected  distribution 


model  order  increases  due  to  the  product  automaton  con¬ 
struction.  On  the  other  hand,  the  projective  composition 
has  the  same  number  of  states  as  the  second  argument  in 
(•)&(•).  Both  representations  have  exactly  the  same  pro¬ 
jected  distribution  with  respect  to  a  fixed  second  argument, 
thus  making  ®  an  extremely  useful  tool  for  model  order 
reduction.  Algorithm  [3]  computes  the  projected  composition 
of  two  arbitrary  PFSA. 


Algorithm  3:  Computation  of  Projected  Composition 

input  :  P, j=  (Q,  Z,  6,  qi:  ft),  Gy  =  (Q',  Z,  5',  qy ,  ft') 
output:  Pj®Gy 

1  begin 

2  Compute  Pi  ig>  Gy  =  (Q  x  Q',  Z,  6®,  (qlr  (/',),  ft®); 

3  /*See  Definition  rot/ 

4  Compute  p;  /*  State  Prob.  for  P;®G;,  (Def.  HOI  */ 

5  Set  up  matrix  T  s.t.  Tjy  =  p((qj,q'k)); 

6  Compute  ft®  =  Tft; 

7  return  P,-®G,v  =  (Q',  Z,  6',(jj/,ft®); 

8  end 


Denoting  Gy  ®  P,  =  (Q  x  Q',  E,<5®,((/;,  q'it),  ft®)  and  Gy®Pj  = 
(Q,  E,  S,  qi,n®),  we  have  fron  Definition  15.11  that 


L^eQ-  9Wk^cij)n^((q’k„qj),a) 


=  n®({f(qj),qi),o)  =  n'(f(qj),o) 


where  the  last  step  follows  from  Definitionl4.5l  The  proof  is 
completed  by  noting 


Vx  e  E*,  H(G,/)(x)  =  n'lq'^x)  =  n®{(f(qi),qi),x) 
=  n®(qi,x)  =  M(Gy®Pi)(x) 


□ 

Example  5.1:  The  results  of  Section  [5]  are  illustrated  con¬ 
sidering  the  PFSA  models  described  in  Example  \4.1\  Given 
the  PFSA  models  Pf,  =  {\qi,q?_,q3},T,b* l ,q\,  ft1)  and  P2 3 4 5 6 7 8A  = 
(\qA,qB},£.,52,qA,n2)  (See  Eqns.  f40\  and  STD),  we  compute  the 
projected  compositions  P(f8P2A  =  b2,qA,n12)  and 

P^Pji  =  The  synchronous  composi¬ 

tions  Pjj  ®  P2a  and  P2a  ®  P^  were  computed  in  Example  I4.il 
and  are  shown  in  Fig.  [8]  Denoting  the  associated  stochastic 
transition  matrices  for  P^  ®  P2A  and  P2A  ®  P^  as  If12  and  Yl21 
respectively,  we  note: 


0.2000.8 

0.9000.1 

•••(171'‘7a) 

00.3.700 

00.9.100 

•  •  •  (qi,  qA) 

.4000.60 

,n21  = 

.9000.10 

■  ■  ■  0?3,  qA) 

0.2000.8 

0.7000.3 

•••(‘?u‘?b) 

00.3.700 

00.7.300 

•  •  •  (?2,  qf) 

.4000.60 

.7000.30 

•••(1?3/1?b) 

The  stable  probability  distributions  p12  and  p21  are  computed 


to  be: 


B.  Incurred  Error  in  Projective  Composition 

Given  any  two  PFSA  Pj  and  Gy,  the  incurred  error  in 
projective  composition  operation  P®Gy  is  quantified  in  the 
pseudo-metric  defined  in  Section  [4]  as  follows: 

Va,s(Pi,  P i® Gy)  (59) 

Next  we  establish  a  sufficient  condition  for  guaranteeing  zero 
incurred  error  in  projective  composition. 

Theorem  5.4:  For  arbitrary  PFSA  Pj  =  (Q,  E,  5,  qi,  ft)  and 
Gy  =  (Q',  E,  6',  q[,,  ft')  with  corresponding  probabilistic  Nerode 
equivalence  relations  N  and  AP,  we  have 

N  g  AT  =>  VaA(Gy,Gy®Pi)  =  0 

Proof:  N  N'  implies  that  there  exists  a  possibly  non- 
injective  map  /  :  Q  — »  Q'  such  that 

Vx  £  E\  5*(qi/X)  =  qjeQ =>  5*(q'„x)  =  f{qj)  e  Q' 

It  then  follows  from  Definition  15.  II that 
9{q'k,,qj)  =  0ii  f(qj)^q'k, 


p12  =  [0.1458  0.0695  0.0864  0.2017  0.2186  0.2780]  (60a) 

p21  =  [0.2917  0.2917  0.2917  0.0417  0.0417  0.0417]  (60b) 


Using  Algorithm  [3]  we  compute  the  event  generating  func¬ 
tions  fl12  and  fl21  as: 


ft12 


0.7197 

0.6891 


0.2803 

0.3109 


0.1250  0.8750 

0.1250  0.8750 

0.1250  0.8750 


(61) 


We  note  that  the  stable  distributions  for  P^  ®  P2A  and  P2A  ®  P^ 
are  given  by: 


p12  =  [0.3017  0.6983],  p2 


[0.3333  0.3333  0.3333] 


(62) 


The  operations  are  illustrated  in  Figs.  [TO]  and  [77]  and 
invariance  of  the  projected  distribution  is  checked  as  follows: 

P12(  1)  +  so12  (2)  +  sj12(3)  =  0.3017  =  ^(1)  (63a) 

p12(  4)  +  jo12  (5)  +  sj12(6)  =  0.6983  =  p%)  (63b) 
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P 21  (2)  +  p21  (5)  =  0.333  =  p21  (2)  (63c) 

p21  (3)  +  p21  (6)  =  0.333  =  (3)  (63d) 


6.  An  Engineering  Application  of  Pattern 
Recognition 

Projective  composition  is  applied  to  a  symbolic  pattern 
identification  problem.  Continuous-valued  data  from  a  laser 
ranging  array  in  a  sensor  fusion  test  bed  are  fed  to  a 
symbolic  model  reconstruction  algorithm  (CSSR)  [1]  to  yield 
probabilistic  finite  state  models  over  a  four-letter  alphabet. 
A  maximum  entropy  partitioning  scheme  [3]  is  employed 
to  create  the  symbolic  alphabet  on  the  continuous  time 
series.  Figure  [12]  depicts  the  results  from  four  different 
experimental  runs.  Two  of  those  runs  in  the  top  two  rows 
of  Fig.  [12]  correspond  to  a  human  subject  moving  in  the 
sensor  field;  the  other  two  runs  in  the  bottom  two  rows  corre¬ 
spond  to  a  robot  representing  an  unmanned  ground  vehicle 
(UGV).  The  symbolic  reconstruction  algorithm  yields  PFSA 
having  disparate  number  of  states  in  each  of  the  above  four 
cases  (i.e.,  two  each  for  the  human  subject  and  the  robot), 
with  their  graph  structures  being  significantly  different.  The 
resulting  patterns  (i.e.,  state  probability  vectors)  for  these 
PFSA  models  in  each  of  the  four  cases  are  shown  on  the  left 
side  of  Fig.  [12]  The  models  are  then  projectively  composed 
with  a  64  state  D-Markov  machine  [2]  having  alphabet  size 
=  4  and  depth  =  3.  The  resulting  pattern  vectors  are  shown 
on  the  right  hand  column  of  Fig.  1121  The  four  rows  in  Fig.  [12] 
demonstrate  the  applicability  of  projective  composition  to 
statistical  pattern  classification;  the  state  probability  vectors 
of  projected  models  unambiguously  identify  the  respective 
patterns  of  a  human  subject  and  an  UGV. 


7.  Summary,  Conclusions  &  Future  Work 

This  paper  presents  a  rigorous  measure-theoretic  ap¬ 
proach  to  probabilistic  finite  state  machines.  Key  concepts 
from  classical  language  theory  such  as  the  Nerode  equiva¬ 
lence  relation  is  generalized  to  the  probabilistic  paradigm 
and  the  existence  and  uniqueness  of  minimal  represen¬ 
tations  for  PFSA  is  established.  Two  binary  operations, 
namely,  probabilistic  synchronous  composition  and  projective 
composition  of  PFSA  are  introduced  and  their  properties 
are  investigated.  Numerical  examples  have  been  provided 
for  clarity  of  exposition.  The  applicability  of  the  defined 
binary  operators  has  been  demonstrated  on  experimental 
data  from  a  laboratory  test  bed  in  a  pattern  identification 
and  classification  problem.  This  paper  lays  the  framework  for 
three  major  directions  for  future  research  and  the  associated 
applications. 

•  Probabilistic  Non-regular  Languages:  Since  projec¬ 
tive  composition  can  be  used  to  obtain  smaller  order 
models  with  quantifiable  error,  the  possibility  of  pro¬ 
jectively  composing  infinite  state  probabilistic  models 
with  finite  state  machines  must  be  investigated.  The 
extension  of  the  theory  developed  in  this  paper  to  non¬ 
regular  probabilistic  languages  would  prove  invaluable 
in  handling  strictly  non-Markovian  models  in  the  sym¬ 
bolic  paradigm,  especially  physical  processes  that  fail  to 
have  the  semi-Martingle  property,  e.g.,  fractional  Brown¬ 
ian  motion  [19].  Future  work  will  investigate  language- 
theoretic  non-regularity  as  the  symbolic  analogue  to 
chaotic  behavior  in  the  continuous  domain. 

•  Optimal  Control:  The  reported  measure-theoretic  ap¬ 
proach  to  optimal  supervisor  design  in  PFSA  models  will 
be  extended  in  the  light  of  the  developments  reported  in 
this  paper  to  situations  where  the  control  specification  is 
given  as  weights  on  the  states  of  DFSA  models  disparate 
from  the  plant  under  consideration.  Such  a  general¬ 
ization  would  allow  the  fusion  of  Ramadge  and  Won- 
ham’s  constraint  based  supervision  approach  [8]  with 
the  measure-theoretic  approach  reported  in  [10][11]. 
This  new  control  synthesis  tool  would  prove  invaluable 
in  the  design  of  event  driven  controllers  in  probabilistic 
robotics. 

•  Pattern  Identification:  Preliminary  application  in  pat¬ 
tern  classification  has  already  been  demonstrated  in 
Section  [6]  Future  research  will  formalize  the  approach 
and  investigate  methodologies  for  optimally  choosing  the 
plant  model  on  which  to  project  the  constructed  PFSA 
to  yield  maximum  algorithmic  performance.  Future  in¬ 
vestigations  will  explore  applicability  of  the  structural 
transformations  developed  in  this  paper  for  the  fusion, 
refinement  and  computation  of  bounded  order  symbolic 
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Fig.  12.  Experimental  Validation  of  Projective  Composition  in  Pattern  Recognition:  (a)  and  (b)  correspond  to  ranging  data  for  a  human 
subject  in  sensor  field;  (c)  and  (d)  correspond  to  an  UGV 


models  of  observed  system  behavior  in  complex  dynam¬ 
ical  systems. 
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Supervisory  control  theory  for  discrete  event  systems,  introduced  by  Ramadge  and  Wonham, 
is  based  on  a  non-probabilistic  formal  language  framework.  However,  models  for  physical 
processes  inherently  involve  modelling  errors  and  noise-corrupted  observations,  implying 
that  any  practical  finite-state  approximation  would  require  consideration  of  event  occurrence 
probabilities.  Building  on  the  concept  of  signed  real  measure  of  regular  languages,  this  paper 
formulates  a  comprehensive  theory  for  optimal  control  of  finite-state  probabilistic  processes. 
It  is  shown  that  the  resulting  discrete-event  supervisor  is  optimal  in  the  sense  of  elementwise 
maximizing  the  renormalized  langauge  measure  vector  for  the  controlled  plant  behaviour  and 
is  efficiently  computable.  The  theoretical  results  are  validated  through  several  examples  includ¬ 
ing  the  simulation  of  an  engineering  problem. 


1.  Introduction 

Supervisory  control  theory  (SCT)  of  discrete-event 
systems  (DES),  pioneered  by  Ramadge  and 
Wonham  (1987),  models  a  physical  or  human-engi¬ 
neered  process  as  a  finite-state  language  generator  and 
constructs  a  supervisor  that  attempts  to  constrain  the 
“supervized”  plant  behaviour  within  a  specification  lan¬ 
guage.  The  original  theory  is  based  on  a  deterministic 
language  framework.  Although  allowing  non-determin¬ 
ism  in  the  sense  that  more  than  one  continuation  of  a 
generated  event  trace  (i.e.,  a  string)  is  possible,  no 
attempt  is  made  to  quantify  this  randomness. 
As  Wonham  himself  observes  in  Lawford  and 
Wonham  (1993),  “the  choice  of  a  possible  continuation 
of  a  string  is  made  by  some  internal  structure  unmodeled 
by  the  systems  designer”.  The  notion  of  probabilistic 
languages  in  the  context  of  studying  qualitative  stochas¬ 
tic  behaviour  of  discrete-event  systems  first  appears 
in  Garg  (1992a,  b),  where  the  concept  of  p-languages 
(‘p’  implying  probabilistic)  is  introduced  and  an  algebra 
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is  developed  to  model  probabilistic  languages  based  on 
concurrency  (Milner  1989).  A  regular  p-language  is 
essentially  a  set  of  prefix-closed  traces  of  events, 
generated  by  a  finite-state  automaton  with  probabilities 
associated  with  the  transitions.  A  p-language-theoretic 
model  differs  in  several  important  aspects  from 
other  discrete-event  models  of  stochastic  analysis 
such  as  Markov  chains  (Cassandras  and  Lafortune 
1999),  stochastic  Petri  nets  (Molloy  1982,  Chung  et  al. 
1994),  probabilistic  automata  (Rabin  1963,  Paz  1971, 
Doberkat  1981),  and  fuzzy  models  (Lee  and 
Zadeh  1969).  Garg  et  al.  (1999)  and  Kumar  and  Garg 
(2001)  provide  a  brief  comparison  of  the  p-language- 
theoretic  modelling  paradigm  with  the  above-mentioned 
theories. 

Lawford  and  Wonham  (1993)  have  attempted  to 
extend  discrete-event  (SCT)  to  plants  modelled  by 
p-languages,  where  a  formal  statement  of  the  probabilis¬ 
tic  supervisory  control  problem  (PSCP)  first  appears  and 
the  notion  of  probabilistic  supervision  is  introduced  by 
random  disabling  of  controllable  events.  The  key  differ¬ 
ence  from  other  stochastic  supervision  approaches  (e.g., 
Mortzavian  1993)  lies  in  the  fact  that  the  computed 
probabilistic  supervisor  is  not  allowed  to  change  the 
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underlying  plant  dynamics  in  the  following  sense: 
“The  probabilistic  effect  of  random  disablement  is  deter¬ 
mined  entirely  by  the  plant”.  The  control  objective  is 
specihed  as  a  p-language  and  necessary  and  sufficient 
conditions  are  derived  for  existence  of  a  probabilistic 
supervisor  that  attempts  to  restrict  the  plant  language 
within  the  control  specification  in  a  probabilistic  sense. 
The  theory  of  supervision  of  p-languages  is  further 
developed  by  Kumar  and  Garg  (2001),  where  the  control 
objective  is  specified  as  upper  and  lower  bound 
constraints.  The  upper  bound  is  a  non-probabilistic 
language  that  serves  as  a  legality  constraint,  while  the 
lower  bound  is  a  p-language.  This  relatively  relaxed 
approach  to  control  objective  specification  allows  for  a 
non-probabilistic  supervisor  that  attempts  to  cut  down 
illegal  event  traces,  while  ensuring  that  legal  traces 
occur  with  probabilities  greater  than  or  equal  to  that 
specihed  by  the  lower  bound.  Intuitively,  the  designed 
supervisor  stops  “bad"  strings  from  occurring  while 
guaranteeing  that  “good”  strings  occur  with  some  mini¬ 
mum  pre-set  frequency.  However,  construction  of  such 
a  control  objective  specification  may  not  be  possible  in 
many  applications  (e.g.,  battlefield  command,  control, 
communications,  and  intelligence  (C3I)  (Phoha  et  al. 
2002)),  especially  if  the  decisions  are  to  made  in  real 
time.  For  the  theory  to  be  useful  in  practice,  one 
must  generate  the  specification  from  the  definition  of 
the  physical  problem  at  hand.  Given  that  one  has 
to  come  up  with  a  non-probabilistic  language  to  serve 
as  the  upper  legality  constraint  and  a  probabilistic 
language  to  serve  as  the  lower  bound,  this  goal 
may  not  always  be  achievable.  The  situation  becomes 
worse  for  non-stationary  stochastic  environments, 
where  the  control  specifications  may  have  to  be  updated 
online. 

A  significantly  simplified  approach  to  the  above  pro¬ 
blem  is  reported  by  Ray  (2005)  and  Ray  et  al.  (2005), 
where  the  control  objective  is  specified  as  characteristic 
weights  on  the  states  of  the  plant  automaton.  These 
weights  are  normalized  in  the  interval  [—1,1]  with  posi¬ 
tive  weights  assigned  to  good  states  and  negative  weights 
to  bad  states.  A  signed  real  measure  of  regular  languages 
(of  event  traces)  is  defined  as  a  function  of  the  character¬ 
istic  weights  and  the  state  transition  probabilities;  and 
supervisory  control  laws  are  synthesized  by  elementwise 
maximizing  the  language  measure  vector  (Ray  et  al. 
2004,  2005).  Intuitively,  the  supervisor  ensures  that  the 
generated  event  traces  cause  the  plant  to  visit  the 
“good”  states  while  attempting  to  avoid  the  “bad” 
states  in  a  probabilistic  sense.  As  mentioned  earlier, 
Kumar  and  Garg’s  work  on  supervisory  control  of 
probabilistic  automata  (Kumar  and  Garg  2001)  also 
has  a  notion  of  “good”  and  “bad”  strings.  However, 
the  classification  is  strictly  binary;  the  theory  has  no 
way  of  saying  if  one  “good”  string  is  “better”  than 


another  “good”  string  and  vice  versa.  This  implies  that 
the  supervisor  must  eliminate  all  bad  strings  and  hence 
may  not  be  optimal,  or  fail  to  exist  if  the  conditions 
defined  in  Kumar  and  Garg  (2001)  are  not  satisfied. 
In  contrast,  in  the  measure-theoretic  approach 
(Ray  2005),  the  event  traces  are  less  or  more  desirable 
in  a  continuous  scale  with  the  supervisor  optimizing 
the  controlled  plant  behaviour  to  ensure  that  the 
“most”  desirable  strings  occur  “most”  often.  This  has 
an  immediate  advantage  that  the  problem  of  existence 
disappears;  the  optimal  supervisor  always  exists  and 
can  be  computed  effectively  with  polynomial  complex¬ 
ity.  The  latter  approach  is,  in  one  sense,  closer  to 
Markov  chain  modelling  since  the  control  specification 
is  state-based.  However,  as  shown  by  Ray  (2005), 
this  does  not  restrict  the  modeling  power  of  the  techni¬ 
que.  It  is  shown  in  Kumar  and  Garg  (2001)  that, 
in  general,  maximally  permissive  supervisors  are 
non-unique.  For  the  measure-theoretic  approach,  how¬ 
ever,  the  optimization  is  shown  to  yield  unique  maximal 
permissiveness  among  all  optimal  supervisors  (Ray  et  al. 
2004,  2005). 

Optimal  control  in  the  context  of  discrete  event 
dynamic  systems  has  been  addressed  earlier  by  several 
investigators  as  cited  in  (Ray  et  al.  2004).  For  example, 
Sengupta  and  Lafortune  (1998)  have  analysed 
non-probabilistic  DES  with  assigned  event  and  control 
costs;  the  optimal  supervisor  is  computed  in  the  frame¬ 
work  of  dynamic  programming  (DP)  with  two  critical 
assumptions  to  guarantee  polynomial  complexity  of 
the  solution:  all  costs  are  strictly  positive  and  there  is 
only  one  marked  state  (Sengupta  and  Lafortune  1998, 
p.  34).  The  work  reported  in  Ray  (2005)  and  Ray 
et  al.  (2005)  is  different  in  the  sense  that  the  latter 
deals  with  probabilistic  automata  and  the  optimization, 
even  in  the  completely  general  case,  has  guaranteed 
polynomial  complexity  of  0(n3),  where  n  is  the  number 
of  states  in  the  unsupervised  plant  model.  The  mea¬ 
sure-theoretic  approach  was  originally  reported  for  a 
restricted  class  of  terminating  p-languages  Ray  (2005) 
and  Ray  et  al.  (2005);  and  this  restriction  has  been  elimi¬ 
nated  in  a  subsequent  publication  (Chattopadhyay  and 
Ray  2006a). 

The  notion  of  terminating  and  non-terminating  auto¬ 
mata  is  originally  due  to  Garg  (19992a,  b).  A  probabil¬ 
istic  automaton  is  terminating  if  there  exist  states  at 
which  the  sum  of  the  probabilities  of  all  defined  events 
is  strictly  less  than  1 .  The  interpretation  is  that  the  differ¬ 
ence  of  the  sum  from  1  is  the  probability  that  the  plant 
terminates  operation  at  that  particular  state.  It  is  shown 
in  Ray  (2005)  that  the  language  measure  vector  can  be 
expressed  as  [D  —  II]- 1  x  where  n  is  the  transition  prob¬ 
ability  matrix  and  x  is  the  characteristic  vector,  where 
Ttjj  is  the  probability  of  transition  from  the  r'th  state  to 
the  /th  state  and  xi  is  the  characteristic  weight  of  the 


Optional  control  of  probabilistic  finite-state  systems 


1273 


state  i).  A  sufficient  condition  for  the  inverse  of  [l  —  II] 
to  exist  is  that  fj.  Tty  <1  e  i,  i.e.,  the  plant  has  a  strictly 
non-zero  probability  of  termination  from  each  state. 

This  paper  eliminates  the  above  restrictive  assumption 
by  adopting  the  recently  reported  renormalized  measure 
of  regular  languages  (Chattopadhyay  and  Ray  2006a) 
as  the  performance  index.  It  also  extends  the  measure- 
theoretic  concept  for  optimal  control  of  terminating 
plants  (Ray  et  al.  2004)  to  non-terminating  plant 
models,  which  requires  a  minor  modification  of  the 
control  philosophy  as  explained  below. 

Supervisors  in  the  SCT  paradigm  are  allowed  to  affect 
the  underlying  plant  behaviour  by  selectively  disabling 
controllable  events  (Ramadge  and  Wonham  1987). 
In  case  of  terminating  p-languages,  a  similar  approach 
suffices;  the  supervisor  selectively  nulls  the  occurrence 
probability  of  controllable  events  to  achieve  the  desired 
control  objective.  However,  the  non-terminating  case 
poses  a  problem  since  any  such  disabling  action  converts 
the  system  to  a  terminating  p-language  (i.e.,  the  prob¬ 
abilities  of  the  events  defined  at  a  state  fail  to  add  up 
to  1).  The  solution  (Kumar  and  Garg  2001)  is  to  propor¬ 
tionately  increase  the  probabilities  of  the  remaining 
enabled  events  at  the  state  at  which  event  disabling  is 
undertaken.  An  alternative  approach  is  proposed  in 
this  paper,  where  each  disabled  event  creates  a  self 
loop  at  the  state  (at  which  the  event  was  generated) 
with  occurrence  probability  of  the  original  transition. 

The  paper  is  organized  in  six  sections  and  an 
appendix.  Section  2  lays  down  the  basic  framework  of 
the  analysis  and  briefly  reviews  the  original  notion  of 
language  measure  (Ray  2005)  and  its  renormalization 
(Chattopadhyay  and  Ray  2006a).  Section  3  formulates 
the  optimal  control  problem  based  on  the  concept  of 
renormalized  measure  and  presents  the  key  results. 
Section  4  presents  a  solution  of  the  optimal  control 
problem  and  derives  the  necessary  algorithms  for  its 
implementation.  Section  5  presents  an  engineering 
example,  where  the  optimal  supervisor  is  designed  for 
a  three-processor  message  decoding  system.  The 
paper  is  summarized  and  concluded  in  §6  along  with 
recommendations  for  future  research.  Appendix  G 
establishes  bounds  on  the  derivatives  of  the 
renormalized  measure  that  is  necessary  for  formulation 
of  the  optimal  control  law  in  §3. 


2.  Preliminary  concepts 


2.1  Brief  review  of  language  measure 

Let  the  plant  behaviour  be  modelled  as  a  deterministic 
finite  state  automaton  (DFSA)  as  G;  =  ( Q ,  E,  8,  qt,  Qm), 
where  Q  is  the  finite  set  of  states  with  \Q\  —  n,  and 
cpeQ  is  the  initial  state;  E  is  the  (finite)  alphabet  of 
events  with  |E|  =  m;  the  Kleene  closure  of  E  is  denoted 
as  E*  that  is  the  set  of  all  finite-length  strings  of  events 
including  the  empty  string  e;  the  (possibly  partial)  func¬ 
tion  8 :  Q  x  E  — >  Q  represents  state  transitions  and 
8*:  Q  x  E*  -»  Q  is  an  extension  of  <5;  and  Qm  c  Q  is 
the  set  of  marked  (i.e.,  accepted)  states. 

Definition  1:  The  language  L(  G;j  generated  by  the 
DFSA  Gi  is  defined  as  L(Gj)  =  {s  e  E*  |  8*{qhs)  e  Q}. 

Definition  2:  The  marked  language  L„,(G,)  by  the 
DFSA  G/  is  defined  as  Lm(G f)  —  {s  e  E*|  8*{qt,s)  e  Qm}. 

The  language  L(Gj)  of  the  DFSA  G,  is  partitioned  as 
the  non-marked  and  the  marked  languages, 
L°(Gj)  =  L(Gj)  —  L„,(G,j  and  L,„(G,),  consisting  of  event 
strings  that,  starting  from  qt  e  Q ,  terminate  at  one  of 
the  non-marked  states  in  Q  —  Qm  and  one  of 
the  marked  states  in  Qm,  respectively.  The  set  Qm  is 
partitioned  into  Qfn  and  Q~  where  Qfn  contains  all 
good  marked  states  that  one  may  desire  to  terminate 
on,  and  Q~  contains  all  bad  marked  states  that  one 
would  attempt  to  avoid  terminating  on,  although  it 
may  not  always  be  possible  to  bypass  a  bad  state 
before  reaching  a  good  state.  The  marked  language 
Lm(G,)  is  further  partitioned  into  L+(G,)  and  L“(Gj) 
consisting  of  good  and  bad  strings  that,  starting  from 
cji,  terminate  on  Q+  and  Q~,  respectively. 

A  signed  real  measure  /x:  2s*  — >  R  =  (— oo,  oo)  is 
constructed  for  quantitative  evaluation  of  every  event 
string  s  e  E*.  The  language  L(  Gj)  is  decomposed  into 
null,  i.e.,  L°(Gj),  positive,  i.e.,  L+(Gjj,  and  negative, 
i.e.,  L~(Gj)  sublanguages. 

Definition  3:  The  language  of  all  strings  that,  starting 
at  a  state  qt  e  Q,  terminates  on  a  state  qj  e  Q ,  is  denoted 
as  L(qi,  qfi.  That  is, 

L(‘ 1i,  qj)  =  {*€  L(Gi)  ■  8*{qi,  s)  =  qj}.  (1) 


Definition  4:  The  characteristic  function  that  assigns  a 
signed  real  weight  to  each  state  cp,i  —  1,2,...,  n,  is 
defined  as:  x'-  Q  [—  b  1]  such  that 


This  section  briefly  reviews  the  concept  of  signed  real  [—  1,  0)  if  cp  e  Q~ 

measure  of  regular  languages  Ray  (2005)  and  Ray 

et  al.  (2005)  followed  by  a  review  of  the  notion  of  xioj) e  W  if  qj^Qm 

renormalized  measure  and  the  pertinent  notations  used 
in  the  sequel. 


(0,1]  if  qj  e  Qj 
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Definition  5:  The  event  cost  is  conditioned  on  a  DFSA 
state  at  which  the  event  is  generated,  and  is  defined  as 
it :  E*  x  Q  ->  [0,  1]  such  that  Vqj  e  g,V< t/c  e  E, 
Vs  e  E\ 

1.  7t[ak,qj\  =  Ttjk  e  [0, 1);  Yk^jk  <  f 

2.  Tt[a,qj\  —  0  if  8{qj,a)  is  undefined;  Tt[e,qj\  =  1; 

3.  n[aks,  qj]  —  Tr[ak,  qj  n[s,  8(qh  ak)\. 

The  event  cost  matrix,  denoted  as  fl-matrix,  is 
defined  as 


7I\\ 

.  .  TC \m 

^n\ 

•  •  Tlnm 

An  application  of  the  induction  principle  to  part  (3)  in 
Definition  5  shows  jr[st,  qj  —  tt[s,  qjn[t,  8*{qj,  s)]. 
The  condition  T,kn jk  <  1  provides  a  sufficient  condition 
for  the  existence  of  the  real  signed  measure  as  discussed 
in  Ray  (2005)  along  with  additional  comments  on  the 
physical  interpretation  of  the  event  cost. 

Now  let  us  define  the  measure  of  a  sublanguage  of  the 
plant  language  L(Gj  in  terms  of  the  signed  characteristic 
function  /  and  the  non-negative  event  cost  n. 

Definition  6:  The  signed  real  measure  p  of  a  singleton 
string  set  {5}  c  L(qh  qj)  c  L(Gj  e  2E*  is  defined  as 

p({s})  =  qjxiqj  V.y  e  L(qt,  qj. 

The  signed  real  measure  of  L(qh  qj)  is  defined  as 

/x(L(</„  qj)  =  MW) 

seUqi,  qj) 


and  the  signed  real  measure  of  a  DFSA  G;,  initialized  at 
the  state  qt  e  Q ,  is  denoted  as 

Pi  =  p(L(Gj)  -  Yj  d{L(di,  qj)- 

Definition  7:  The  state  transition  cost  of  the  DFSA  is 
defined  as  a  function  it:  Q  x  Q  -*  [0,  1)  such  that 

Mqj-  q/j 

0  if  {er  e  E :  8{qj,  a)  —  qk)  —  0 

—  jr(cr,  qj  =  Ttjk  otherwise 

o€Y,-.8(qj,a)=qk 

(2) 


It  has  been  shown  in  (Ray  2005  and  Ray  et  al.  2005) 
that  the  measure  pt  =  p(L(Gj)  of  the  language  L(Gj, 
with  the  initial  state  qj,  can  be  expressed  as 
di  —  Fj  +  Xi  where  Xi-x(qd ■  Equivalently,  in 
vector  notation 

p  =  np  +  x  /i  =  [O-nr'jc,  (3) 

where  the  measure  vector  p  =  [p\  P2  p,,]1  and 
the  characteristic  vector  x  —  [xi  Xi  Xn]T',  and  the 
condition  Yj^ij  <1  si  (see  Definition  5)  is  sufficient 
for  the  inverse  to  exist. 

Although  the  preceding  analysis  reported 
in  (Ray  2005  and  Ray  et  al.  2005)  was  intended  for 
non-probabilistic  regular  languages,  the  event  costs  can 
be  easily  interpreted  as  occurrence  probabilities. 
As  such  the  fl-matrix  is  analogous  to  the  morph 
matrix  of  a  Markov  chain  in  the  sense  that  an  element 
jtj  represents  the  probability  of  the  /th  event  occurring 
at  the  zth  state  with  the  exception  that  the  strict  inequal¬ 
ity  condition  Yj  Ty  <  1  is  enforced  instead  of  satisfying 
the  equality.  Equivalently,  the  II-matrix  is  analogous  to 
the  state  transition  probability  matrix  of  a  Markov 
chain  in  the  sense  that  an  element  7 xjk  is  analogous  to 
the  transition  probability  from  state  qj  to  state  qk  with 
the  exception  that  the  strict  inequality  condition 
Yk^jk  <  1  *s  enforced  instead  of  satisfying  the 
equality.  It  follows  that  the  preceding  analysis  is  applic¬ 
able  to  the  case  of  terminating  probabilistic  languages 
(Garg  et  al.  1992a,  b)  that  have  a  strictly  non-zero 
probability  of  termination  at  each  state. 

Let  E"  denote  the  set  of  all  unmodelled  events  in  the 
terminating  plant.  A  new  unmarked  absorbing  state 
qn+\,  called  the  dump  state  (Ramadge  and  Wonham 
1987),  is  created  and  the  transition  function  8  is  extended 
to  <5ext  :  ( Q  U  {<?„+[ })  x  (E  U  E")  -*  (Q  U  {qn+x }).  The 
residue  Oj  —  1  —  Yk 71 'jk  denotes  the  probability  of  transi¬ 
tion  from  qj  to  q„+\.  Consequently,  the  II-matrix 
(see  Definition  7)  is  augmented  to  obtain  the  stochastic 
state  transition  probability  matrix  as 


7t\\ 

7Ti2 

...  7t\n 

01- 

Jt2l 

7T22 

...  7t2n 

02 

(4) 

ktjil 

Xn2 

. .  .  J Xnn 

On 

_  0 

0 

0  ... 

1  _ 

The  state  transition  cost  matrix,  denoted  as  II-matrix,  is 
defined  as 


7l\\ 

71 1 11 

Jtn\ 

.  .  7Tnn 

Since  the  dump  state  qn+x  is  not  marked  (Ramadge  and 
Wonham  1987),  it  follows  from  Definition  4  that  the 
corresponding  state  weight  x„+\  —  0-  Hence,  the 
/-vector  is  augmented  as 


n  = 


Xaug  =  Yir  0]'- 


(5) 
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Denoting  ©  =  [6{  02  ■  ■  ■  dn]T,  where  0,- e  (0, 1)  is  the 
probability  of  transition  from  the  state  q,-  to  the  dump 
state,  it  follows  from  equations  (4)  and  (5)  that  the 
measure  of  the  augmented  system  (Chattopadhyay  and 
Ray  2006a)  is 


A*aug(®)  =  M®f  Of .  (6) 

Then,  the  event  cost  matrix  (see  Definition  5)  and  the 
state  transition  cost  matrix  (see  Definition  7)  can  be 
represented  as 


fi(0)  = 


D  -  Diag[0] 


P 


and 


n(0)  = 


D  -  Diag[0] 


P. 

(7) 


vector  x  becomes  one-dimensional  and  can  be  assigned 
as  x  —  1;  consequently,  the  measure  vector  /i(0) 
degenerates  to  the  scalar  measure  0”1.  To  alleviate  the 
singularity  of  the  matrix  operator  [I  — 11(0)]  at  0  =  0, 
the  measure  vector  fiiO)  in  (3)  is  normalized  with  respect 
to  0_1  to  obtain  the  renormalized  measure  vector. 

Definition  8:  The  renormalized  measure  is  defined  as 

v(0)  =  0  p(P)  =  0  [D  -  n(0)]_1x  (10) 

and  it  follows  from  (8)  that 


0  IhvgiO)  =  M0)r  0]r  (11) 


where  P  and  P  are  both  stochastic  matrices  (Bapat  and 
Raghavan  1997),  i.e.,  EyPy  =  1  V/  e  {1, . . . ,  m]  and 
XjPij=  1  Vie  {1  — ,«}. 

If  the  probability  of  termination  (or  equivalently  the 
probability  of  transition  to  the  dump  state)  is  equal 
for  all  states,  cp  e  Q ,  i.e.,  0,-  =  0  Vi  e  {1, 2, ... , ;?},  then 
equation  (6)  is  expressed  as 

ftmgP)  =  IKdf  Of  (8) 

Consequently,  IT  and  II  in  equation  (7)  are  represented  as 

fi(0)  =  (1  -  0)P  and  n(0)  =  (1  -  0)P  (9) 

where  0  is  the  uniform  probability  of  termination  at  all 
states;  and  both  P  and  P  retain  the  properties  of  stochas¬ 
tic  matrices  (Bapat  and  Raghavan  1997). 

2.2  Renormalization  of  language  measure 

The  notion  of  language  measure  has  been  recently 
extended  to  non-terminating  models  by  assuming  a  uni¬ 
form  non-zero  probability  of  termination  (0)  at  each 
state,  renormalizing  the  language  measure  vector  with 
respect  to  the  probability  of  termination  and  computing 
the  limit  of  the  renormalized  measure  (Chattopadhyay 
and  Ray  2006a)  as  0  — »■  0+.  As  the  probability  of  termi¬ 
nation  approaches  zero  (i.e.,  0  — >■  0+),  and  the  plant 
coincides  with  the  desired  non-terminating  model  in 
the  limit.  The  construction  of  renormalized  measure  is 
briefly  outlined  below. 

The  regular  language  generated  by  the  DFSA  under 
consideration  is  a  sublanguage  of  the  Kleene  closure 
E*  of  the  alphabet  E,  for  which  the  automaton  states 
can  be  merged  into  a  single  state.  In  that  case,  the 
state  transition  cost  matrix  11(0)  degenerates  to  the 
1  x  1  matrix  [1—0]  and  the  normalized  state  weight 


3.  Optimal  control  problem:  formulation 

The  following  notations  are  needed  for  elementwise 
comparison  of  finite-dimensional  vectors  and  matrices 
for  the  analysis  developed  in  the  sequel. 

Notation  1:  Let  V"  and  Vh  be  (m  x  n)  real  matrices. 
The  following  elementwise  equality  and  inequalities 
imply  that 

(WEf)  (k  =  V*j  j  Vi  e  {1, . . .  ,n)  V/  e  {1, . .  .,m} 

(We  V'j  &  (v°.f  Jf  j  3ie{l,...,«},ye{l,...,w} 

(WEC^  (  f'7  >  If)  Vi  e  {1, . . . ,/?}  Y;  e  {1, . .  .,m} 

(WE  (  y’lj  >  If)  Vi  e  {1, . . . ,«}  Y/e  {1,. . .  ,#m}. 

For  the  terminating  plant,  investigated  in  (Ray  2005 
and  Ray  et  al.  2005),  the  optimal  supervisor  selectively 
disables  controllable  transitions  by  setting  their  occur¬ 
rence  probabilities  to  zero.  This  implies  that  if  IT*  and 
n  are  the  transition  probability  matrices  for  the  opti¬ 
mally  supervised  plant  and  the  unsupervised  plant, 
respectively,  then 

irgEn,  i.e.,  tty  <  Tty. 

Since  for  any  non-trivial  supervisor,  there  is  at  least  one 
disabled  transition  in  the  supervised  plant,  i.e., 

3:',  /  such  that  7r,j  >  0  and  jr*j  —  0 

it  follows  that  if  the  unsupervised  plant  is  non¬ 
terminating,  then  any  non-trivial  supervision  will  result 
in  a  terminating  model.  The  policy  of  Kumar  and 
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Garg  2001  maintains  the  non- termination  property  by 
proportionately  increasing  the  probabilities  of  the 
remaining  enabled  events  at  the  state  at  which 
the  disabling  action  is  applied.  The  first  issue  here  is 
that  the  supervisor  must  be  able  to  affect  the  event 
occurrence  probabilities,  which  is  more  than  just  inhibit¬ 
ing  a  transition.  The  second  issue  is  that  there  is  a 
possibility  of  disabling  all  events  defined  at  a  given 
state  if  all  these  events  are  controllable.  In  that  case, 
the  row  sum  cannot  be  maintained  at  1  as  it  becomes 
strictly  equal  to  zero.  Thus,  it  is  necessary  to  impose 
special  constraints  on  the  unsupervised  plant  to  circum¬ 
vent  this  situation.  This  paper  investigates  an  alternative 
approach  as  described  below. 

Definition  9  (control  philosophy):  Disabling  any 
transition  a  at  a  given  state  q  results  in  reconfiguration 
of  the  automaton  structure  as:  Set  the  self-loop 
8(q ,  <y)  —  q  with  the  occurrence  probability  of  a  from 
the  state  q  remaining  unchanged  in  the  supervised  and 
unsupervised  plants. 

This  is  equivalent  to  adding  a  self-loop  to  the  state  at 
which  the  event  is  being  disabled,  with  the  same  occur¬ 
rence  probability  as  the  disabled  transition. 

Preposition  1:  For  the  control  philosophy  in 
Definition  9,  a  supervised  plant  is  non-terminating  if 
and  only  if  the  unsupervised  plant  is  non-terminating. 

Proof:  The  proof  follows  from  two  lemmas. 

Lemma  1:  Each  row  sum  of  the  Il-matrix  remains 
unchanged  after  supervisory  actions  for  the  control 
philosophy  in  Definition  9. 

Proof:  Let  II  and  FT  be  the  transition  probability 
matrices  for  the  unsupervised  and  supervised  plants, 
respectively.  Let  there  be  exactly  one  disabled  transition, 
in  which  a  (controllable)  event  a  at  state  </,■  is  disabled 
and  let  the  occurrence  probability  of  a  at  state  qt  be  it. 
If  S(qi,  a)  —  qk,  then  it  follows  that 


kth  column 

1 

0  0 

...  o 

0" 

0  ■■■ 

Tt 

—JT 

0 

•«—  z'th  row 

0 

0 

0  ■■■ 

0_ 

implying  JTjrf  —  Vz.  The  proof  follows  by 

induction  on  the  number  of  disabled  events.  □ 


Lemma  2:  Self-loops  cannot  be  disabled. 

Proof:  For  the  control  philosophy  in  Definition  9, 
disabling  a  self-loop  at  any  given  state  causes  regenera¬ 
tion  of  the  self-loop  at  the  same  state  with  identical 
occurrence  probability.  □ 

It  is  evident  from  the  above  two  lemmas  that  each  row 
sum  of  the  reconfigured  Il-matrix  remains  invariant. 
The  proof  of  Proposition  1  is  thus  complete.  □ 

Remark  1:  The  control  philosophy  in  Definition  9  is 
natural  in  the  following  sense.  If  qt  qk,  and  the 
controllable  event  a  is  disabled  at  state  qh  then 
the  sole  effect  of  the  supervisory  action  is  to  prevent 
the  plant  front  making  a  transition  to  the  state  qk. 
That  is,  the  plant  is  forced  to  stay  at  the  original  state 
qt  and  this  is  represented  by  the  additional  self-loop  at 
state  </,  instead  of  the  original  arc  from  qt  to  qk. 

The  notion  of  controllability  is  now  clarified  in  the 
context  of  the  present  paper. 

Definition  10  (controllable  transitions):  For  a  given 
plant,  transitions  that  can  be  disabled  in  the  sense  of 
Definition  9  are  defined  to  be  controllable  transitions 
in  the  sequel. 

In  view  of  Definition  10,  controllability  becomes  state- 
based,  i.e.,  transitions  labelled  by  the  same  event  may  be 
controllable  from  one  state  and  uncontrollable  from 
some  other  state.  This  implies  that  the  event  alphabet 
E  cannot  be  partitioned  into  uncontrollable  and 
controllable  events  sets  as  proposed  in  Ramadge  and 
Wonham  (1987).  Thus,  a  controllable  transition  qt^qk 
refers  to  a  triple  {</,-,  cr,  qk\  and  the  set  of  all  such 
transitions  is  denoted  by  <6. 

3.1  Model  specification 

Plant  models  considered  in  this  paper  are  deterministic 
finite  state  automata  (DFSA)  with  well-defined  event 
occurrence  probabilities.  In  other  words,  the  occurrence 
of  events  is  probabilistic,  but  the  state  at  which  the  plant 
ends  up,  given  a  particular  event  has  occurred,  is 
deterministic.  Furthermore,  no  emphasis  is  laid  on  the 
initial  state  of  the  plant  and  it  is  assumed  that  the 
plant  may  start  from  any  state.  Furthermore,  having 
defined  the  characteristic  state  weight  vector  x,  it  may 
not  be  necessary  to  specify  the  set  of  marked  states, 
because  if  X;  =  0,  then  qt  is  not  marked  and  if  x;  0, 
then  qt  is  marked.  Therefore,  plant  models  with  an 
arbitrary  uniform  termination  probability  0  e  (0, 1), 
i.e.,  Qj  —  6  Vz  e  { 1,2,...,  n],  can  be  completely  specified 
by  a  sextuple  as 

G(0)  =  (G,S,5,n(0),x,r^),  (12) 
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where  11(0), ^  is  the  occurrence  probability  of  event  er;-  from 
state  cp  and  11(0), y  =1—0  Vz.  An  application  of  (7) 
with  uniform  uniform  termination  probability  0  yields 
an  alternative  representation  of  the  sextuple  in  (12). 

G(0)=(Q,E,5,(  1-0)P,X,«’),  (13) 

where  P  is  the  the  morph  matrix  of  the  underlying 
Markov  chain. 

As  0  -»  0+,  the  resulting  non-terminating  plant  model 
is  denoted  as 


G(0)  =  (2,E,«5,P,x,^).  (14) 


Definition  11:  Given  0  e  (0, 1),  a  terminating  plant 
G(0)  =  ((?,£,< 5,  ( 1  —  0)P,  x,  ^?)  is  defined  to  be  the 
0-neighbour  of  the  non-terminating  plant 
G(0)  =  (g,E,  *,P,x,«). 

For  a  given  non-terminating  plant  G(0)  and  a  fixed 
0o  e  (0, 1),  there  is  exactly  one  0o-neighbour  G(0o). 

Notation  2:  Let  0  e  (0, 1)  be  the  unform  probability  of 
termination  for  a  terminating  plant  G(0)  =  (Q,  E,  S, 
(1  —  0)P,  x,  f)-  Let  P  be  the  state  transition  probability 
matrix  of  the  underlying  Markov  chain,  which  is  gener¬ 
ated  from  S  and  FI  (see  equation  (2)).  Then,  the  (renor¬ 
malized)  language  measure  vector  (see  Definition  8) 
is  obtained  as 


v(0)  =  0[o-(l-0)p]  ‘x  (15) 

where  (1  —  0)P  is  the  sub-stochastic  transition  probabil¬ 
ity  matrix  for  the  terminating  plant.  Similarly,  for  a  non¬ 
terminating  plant  G(0)  =  (Q,  E,  S,  P,  x,  <&)  having  the 
stochastic  transition  probability  matrix  P,  the  (renorma¬ 
lized)  measure  vector  (Chattopadhyay  and  Ray  2006a)  is 
denoted  as 


v(0)  =  lim  v(0)  =  lim  0 

fl^0+  6^0+ 


(1  -  0)P 


-l 


(16) 


In  the  sequel,  renormalized  measure  v  in  equations  (10) 
and  (11)  is  referred  to  as  measure  for  brevity. 


Definition  12:  For  an  unsupervised  (non-terminating) 
plant  G(0)  =  (Q,  E,  S,  P,  x,  ^),  let  G^  and  G*  be  the 
supervised  plants  with  sets  of  disabled  transitions, 
OF  c  r:(l  and  c  IF,  respectively,  whose  measures  are 
v+  and  v*.  Then,  the  supervisor  that  disables  (f'  is 
defined  to  be  superior  to  the  supervisor  that  disables 
If*  if  yl  SevI  and  strictly  superior  if  v+  >ev*. 

Definition  13  (Optimal  supervision  problem):  Given  a 
(non-terminating)  plant  G(0)  =  ( Q ,  E,  <5,  P,  x,  f),  the 
problem  is  to  compute  a  supervisor  that  disables  a 
subset  c  rF,  such  that 

v*  vf  V#  c  f 

where  v*  and  yl  are  the  measure  vectors  of  the  supervised 
plants  G*  and  (fi  under  SD*  and  respectively. 

Remark  2:  For  a  non-trivial  plant  G(0)  =  (Q,  E,  S, 
P,X,^)  (i.e.,  \Q\  >  1),  there  may  exist  two  supervisors 
that  are  not  comparable  in  the  sense  of  Definition  12. 
For  example,  given  a  two-state  unsupervised  plant  G, 
if  G1'  and  G *  are  supervised  plants  under  two  different 
supervisors  with  disabled  transition  sets,  and  ffi* , 
respectively,  then  the  following  situation  may  occur  for 
the  indices  i  fi  j. 

f  f  A  (»J  <  “1). 

where  v]  and  vj  are  the  z'th  elements  of  the  measure  vec¬ 
tors  for  G'  and  G$,  respectively.  It  is  shown  in  the  next 
section  that,  for  a  given  plant,  an  optimal  supervisor 
(in  the  sense  of  Definition  13)  does  exist  for  which  the 
measure  vector  is  elementwise  greater  than  or  equal 
to  the  measure  vector  of  the  plant  under  any  other 
supervision  policy. 

Terminating  plant  models  have  sub-stochastic 
transition  probability  matrices  (see  Definition  7).  By 
postulating  the  existence  of  unmodelled  transitions, 
such  plants  can  be  transformed  to  non-terminating 
models  as  explained  below.  For  uniform  termination 
probability  0  e  (0, 1),  equations  (8)  and  (11)  suggest 
the  possibility  of  computing  optimal  supervision 
policies  for  terminating  plants  based  on  the  analysis  of 
non-terminating  plants. 


3.2  Construction  of  an  optimal  supervisor 

A  supervisor  disables  a  subset  of  the  set  f  of  controlla¬ 
ble  transitions  and  hence  there  is  a  bijection  between  the 
set  of  all  possible  supervision  policies  and  the  power  set 
2r .  That  is,  there  exists  2|r|  possible  supervisors  and 
each  supervisor  is  uniquely  identifiable  with  a  subset  of 
f  and  the  language  measure  v  allows  a  quantitative 
comparison  of  different  supervision  policies. 


4.  Optimal  control  problem:  solution 

This  section  presents  a  solution  to  the  optimal  supervi¬ 
sion  problem  by  assuming  a  uniform  non-zero  probabil¬ 
ity  of  termination,  0,  at  each  state.  Then,  it  is  shown  that 
the  solution  for  the  corresponding  non-terminating 
plant  can  be  obtained  from  the  control  policy  of  the 
terminating  plant  and  the  bounds  on  the  derivatives  of 
the  language  measure  (see  Appendix  A). 
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Let  0  e  (0, 1)  be  the  uniform  termination  probability  of 
an  unsupervised  plant  G(6)  —  ( Q ,  E,  <5,  (1  —  0) P,  x,  *&). 
The  resulting  (substochastic)  state  transition  cost 
matrix  is  11(0)  =  (1  —  0)P.  For  such  plants  with  uniform 
non-zero  termination  probability,  the  following  lemma 
states  existence  of  an  augmented  plant  model. 

Lemma  3:  For  the  terminating  plant  0(0)  — 
(Q,  E,  8,  (1  —  0)P,  x,  let  the  corresponding 

augmented  non-terminating  plant  be  Gaug  =  (gaug, 
^aug>  ^augi  naug,  Zaug’  ^)-  Let  v*(0)  and  vf(0)  be  the  mea¬ 
sures  of  the  terminating  plant  with  the  respective  sets  of 
disabled  transitions  3*  c  and  3ft  c  r€ .  Then, 

3@*  c  s.t.  v*(0)^ev+(0)  V#  c  %  WOe  (0, 1) 

(17) 


Proof:  It  follows  from  equation  (15)  in  Notation  2  that 

vf  —  v  =  0[n  -  nf]_1-0[D  -  nr‘x 

=  0[d  -  n+]-1([D  -  no  -  [D  -  nt])[D  -  nr  ix 
=  0[o-nt]~‘(nt  -n)v. 

Dehning  the  matrix  A  =  IT^  —  II,  and  the  z'th  row  of  A  as 
A,-,  it  follows  that 

A/rv  (18) 

j  i 

where 


which  implies  that  an  optimal  supervisor  for  Gaug  exists 
(in  the  sense  of  Definition  13)  which  disables  3*  c  . 

Proof:  The  first  n  elements  of  the  measure  vectors  of 
the  augmented  plant  and  the  unaugmented  plant  are 
identically  equal  as  seen  in  equation  (11).  Then,  the 
proof  follows  from  Definition  12.  □ 

The  remainder  of  this  section  derives  an  algorithm  for  a 
supervision  policy  that  elementwise  maximizes 
the  measure  of  the  terminating  plant  G(0).  Lemma  3 
guarantees  that  the  optimal  policy  is  based  on  a  non¬ 
terminating  plant. 

Proposition  2  (Monotonicity):  Let  11(0)  and  v(0)  be  the 
state  transition  cost  matrix  and  the  measure  vector  of 
an  unsupervised  plant  G(0)  =  ( Q ,  E,  8,  (1  —  0)P,  x,  *&), 
respectively.  Let  a  supervisor  be  constructed  to  reconfi¬ 
gure  the  plant  by  disabling  a  set  of  controllable  transi¬ 
tions  3 T  c  such  that  II  is  modified  to  IT1’  by 
following  Algorithm  1.  Then,  denoting  the  measure 
vector  of  the  supervised  plant  by  v’1’,  it  follows  that 
v+  Se  v;  and  equality  holds  if  and  only  if  II+  =  II. 


Algorithm  1:  Monotonlc  Increase  of  Measure  v 


begin 

for  i  -  1  to  n  do 
for  j  =  1  to  n  do 

nf  =  if  +  fit, 
n-;  =  n,7-jS,7 

n?,  =  n v 

nt  =  n„ 

n*=n,7 

i4  =  n»+ft) 

endfor 

endfor 


end 


if  Vj  >  v\  with  Pi j  >  0 


if  Vj  =  Vi 


if  v,  <  Vi  with  pij  >  0 


Ly  — 


{Vj  -  Vj) 
0 

{Vj  -  v,) 


if  Vi  >  vj 

if  vt  =  Vj  Ty  ■  0  WiJ. 

if  Vi  <  Vj 


Since  Y^i=\  Lfy  =  J2'i=\  H-7;  V/',  k,  it  follows  from  non¬ 
negativity  of  II,  that  [/—  II^]-1>eO.  Since  >  0  Vz,  it 
follows  that  A iTv  >  0  Vz  =>v^ev.  For  v,  /  0  and 
A  as  defined  above,  A  J  vk  —  0  if  and  only  if  A  =  0. 
Then,  11^  =  II  and  v7  =  v.  □ 

Corollary  1:  Under  an  identical  situation  to  that 
assumed  in  the  statement  of  Proposition  2,  let  the 
plant  be  reconfigured  as  given  in  Algorithm  2.  Then, 
denoting  the  measure  vector  of  the  modified  plant  by 
v+,  it  follows  that  v^Se  v;  and  equality  holds  if  and 
only  if  nf  =  IT. 

Algorithm  2:  Monotonic  Decrease  of  Measure  v 

begin 

for  ;  -  1  to  n  do 
for  j  =  1  to  n  do 

nf = n;/ + Pi, 

nf = Hu 

nf  =  n,7 

nf  -  n« 

nf  =  n,7-/il7 
nf  =  n,-/  +  Pi, 


endfor 

endfor 

end 


if  Vj  <  Vj  with  p-j  >  0 


|  if  Vj  >  Vj  with  pj  >  0 


Proof:  The  proof  is  similar  to  that  of 
Proposition  26.  □ 

Proposition  2  facilitates  formulation  of  the  algorithm 
for  computing  an  optimal  supervisor  for  plants  with 
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uniform  non-zero  probability  of  termination  at 
each  state.  Let  the  Ath  iteration  of  the  algorithm  com¬ 
pute  a  set  2^]  c  c€  of  controllable  transitions  to  be 
disabled  in  the  sense  of  the  control  philosophy  in 
Definition  9.  The  language  measure  vector  computed 
in  the  k\h  iteration  of  the  algorithm  is  denoted  by 
The  algorithm  terminates  at  the  (&+  l)th  iteration  if 
gfi]  —  g/)[k+i],  which  is  the  optimal  set  of  disabled  transi¬ 
tions  computed  by  the  algorithm  and  is  denoted  by  2*. 
The  algorithm  is  started  with  the  unsupervised  plant 
(i.e.,  with  all  controllable  transitions  enabled)  and 
hence  3r®  =  0.  A  formal  description  is  given  in 
Algorithm  3. 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
21 


Algorithm  3:  Optimal  Supervisor  Algorithm 
input  :  n(0)  =  (1  -  6) P,  x,  ^ 
output:  Optimal  set  of  disabled  transitions  9* 

begin 

SetyW  =  0:  /*  Initial  disabling  set  */ 

Set  k  =  1  ; 

Set  Terminate  =  false  : 
while  (Terminate  ==  false!  do 
Compute  v™  : 

for  j  ml  to  n  do 
for  k  =  1  to  n  do 

Disable  all  conLrollable  transitions  A  tjj 
such  that  Vj  <  v*  ; 

Enable  all  controllable  transitions  < — >  qj 
such  that  vj  i  vj  ; 

endfor 


endfor 

Compute  ; 
if  ==  then 

|  Terminate  =  true 
else 

|  k  =  k  +  1  : 

endif 
endw 

_  g[k\  . 


/*  kfh  disabling  set  */ 


/*  Optimal  disabling  set  */ 


end 


Proposition  3:  Let  be  the  language  measure 
vector  computed  in  the  klh  iteration  of  Algorithm  3. 
The  measure  vectors  computed  by  the  algorithm 
form  an  elementwise  non-decreasing  sequence,  i.e., 
v[A+1]  gE  VM  vfc. 

Proof:  Let  the  state  transition  probability  matrix  in 
the  A:th  iteration  of  Algorithm  3  be  denoted  by  II[A1 
Then,  the  matrix  n[A+1]  is  generated  from  1 by  follow¬ 
ing  the  procedure  as  described  in  Proposition  2.  Hence, 

vt*+‘]  □ 

Proposition  4  (elfectiveness):  Algorithm  3  is  an 
effective  procedure  (Hopcroft  et  al.  2001),  i.e.,  it  is 
guaranteed  to  terminate. 

Proof:  Let  G(0)  —  (Q,  E,  <5,  (1  —  6) P,  x,  W)  be  the  unsu¬ 
pervised  plant  and  let  Card(V)  —  i  e  N.  Denoting  the 


set  of  all  permutations  of  the  vector  [12  ■  ■  ■  l]T  by 
#(£),  a  function  2Y — is  defined  as 


1.  V#C^,(yt  >pt  >...>pt) 

=>(&&)  =  [h  h---in]T) 

2-  (VI  =  yl)  A  (4  >  4) 

=>(if  %(3ft)s  —  4  and  ^(0)t  —  ir  then  s  >  t). 

Let  vM  be  the  measure  vector  computed  in  the  klh  itera¬ 
tion  of  Algorithm  3.  Then,  =  v[Ar+1]  implies  that 
Algorithm  3  terminates  in  k  +  1  iterations  according  to 
its  stopping  rule. 

Next  let  3^  and  3^  be  the  disabling  sets  in  itera¬ 
tions  kx  and  k2,  respectively.  If  then 

v[At,+i]  _  v[*2+i]_  §jnce  £(g)M)  —  it  follows  from 

the  definition  of  £  that  if  vf 1 '  >  then  vf^^Vjkl\ 
If  then  controllable  transitions  qi-^-aqj  are 

disabled  in  both  iterations  k\  +  1  and  k2  +  \ .  If 
jy  —  Vjk'\  then  disabling  or  enabling  controllable 
transitions  q\-+aqj  does  not  affect  the  measure  vector. 
Hence,  it  follows  that  v[A|+1]  and  v[*2+1]  can  be  obtained 
by  disabling  the  same  set  of  controllable  transitions, 
thus  implying  V^I+1' =  v^2+1f  Since  the  measure 
vectors  can  repeat  only  at  the  final  iteration, 
Algorithm  3  is  guaranteed  to  terminate  within 
Card(3(£))  —  i\  iterations.  Therefore,  elfectiveness  of 
Algorithm  3  is  established.  □ 

Next  it  is  established  that  Algorithm  3  is  correct  in  the 
sense  that  an  optimal  supervision  policy  is  generated. 

Proposition  5  (Optimality):  For  a  terminating  plant 
G{&)  —  ( Q ,  £,  S,  (1  -  0)P,  x,  W),  the  supervision  policy 
computed  by  Algorithm  3  is  optimal  in  the  sense  of 
Definition  13. 

Proof:  Let  G(6)  have  the  state  transition  cost  matrix  n, 
measure  v^,  no  disabled  events,  i.e.,  3°  —  0.  Let  G{0)  be 
configured  as  the  supervised  plant  G*(6)  by  application 
of  Algorithm  3  when  it  stops. 

Let  Gr  be  another  configured  plant  distinct  from  G*. 
Let  3*  c  and  f3'  c  9s  be  the  respective  sets  of 
disabled  transitions  and  v*  and  i/  be  the  respective 
measures  for  G*  and  Gl ;  and  3*  /  3k . 

Let  the  following  set  differences  be  denoted  as: 
A  3=3*\3^  and  V®  =  #\$*.  An  application  of 
Algorithm  3  yields 

•  Vz,  j  v*  >  v *  =>  all  controllable  transitions  q ,■  qj  are 
disabled. 

•  V/,  /  i)*Sr*  =>  all  controllable  transitions  q^qj  are 
enabled. 
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To  change  the  plant  configuration  from  G*  to  G^,  all 
transitions  in  AS  are  enabled  and  all  transitions  in 
VS  are  disabled.  Since  any  such  change  requires  us  to 
either  disable  a  transition  qi-^-aqj  where  v*  <  v*  or 
enable  a  disabled  transition  qi-^aqj  where  v *  >  v*,  it 
follows  from  Corollary  1  that  v’1’  =e  v*. 

Since  G ^  is  an  arbitrary  configuration  distinct  from 
G*,  it  follows  that  G*  is  an  optimal  supervision  policy 
in  the  sense  of  Definition  13.  □ 

In  the  reported  work  on  discrete  event  control  of  non- 
probabilistic  regular  languages  (e.g.,  (Ramadge  and 
Wonham  1987)),  the  emphasis  is  on  computing  the 
maximally  permissive  supervisor  in  the  sense  that  the 
supervised  plant  language  is  the  supremal  controllable 
sub-language  of  the  specification.  A  similar  approach 
is  taken  for  probabilistic  regular  languages  (Garg 
1992a,  b).  In  contrast,  the  measure-theoretic  concept 
in  this  paper  computes  a  policy  that  maximizes  the 
elements  of  the  language  measure  vector  elementwise 
to  find  a  supervisor  with  maximal  performance. 
Proposition  5  shows  that  there  exists  at  least  one  optimal 
supervisor.  Now  it  is  shown  that  the  optimal 
supervisor  computed  by  Algorithm  3  is  unique  in  the 
sense  of  being  maximally  permissive  among  all 
policies  that  guarantee  optimal  performance  of  the 
supervised  plant. 

Proposition  6  (uniqueness):  Given  an  unsupervised 
plant  G(0),  the  optimal  supervisor  G*(0),  computed  by 
Algorithm  3,  is  unique  in  the  sense  that  it  is  maximally 
permissive  among  all  possible  supervision  policies  with 
optimal  performance.  That  is,  if  S*  and  S'  are  the  dis¬ 
abled  transition  sets,  and  v*  and  v1'  are  the  language 
measure  vectors  for  G*  and  an  arbitrarily  supervised 
plant  G\  respectively,  then 

V*  =E  V+  S*  c  #  C  c€.  (19) 

Proof:  If  G*  and  Gt  are  distinct,  then  S#  /  S*.  Given 
v*  =e  v#,  let  G*  be  reconfigured  to  G1’  by  disabling  and/or 
re-enabling  appropriate  controllable  transitions.  It 
follows  from  equation  (18)  that 

o  =  vf  —  v*  =  [d  -  rr|  1  (nt  -  ir)v* 

=►  (nt  -  iry  =  o.  (20) 

The  zth  element  of  (IT  —  II*)  v*  is  expressed  as  the 
finite  sum  of  real  numbers 


0  = 


T 


E^ 


r=  1 


(21) 


where  0  <  r  <  2Card ^  and  each  Tr  is  of  the  form: 


T.  = 


a‘r(v*  —  I’*)  >  0,  if  T‘  arises  due  to  disabling 
qi~^  qj  for  some  qj  e  Q 

a‘r(vj  —  v*)  >  0,  if  T‘  arises  due  to  enabling 
q,f  qj  for  some  q}  e  Q 

(22) 


because  each  a‘r  represents  event  occurrence 
probabilities  and  hence  are  positive,  and  the  the  logic 
of  disabling  and  re-enabling  follows  Algorithm  3. 
Therefore,  it  follows  from  equation  (22)  that 
77  =  0Vre{l,...,r}. 

Hence,  it  is  necessary  to  re-enable  controllable  transi¬ 
tions  q [ — >  a  qj  and  disable  the  self  loop  at  q{  such  that 
v\  —  vj  for  reconhguration  from  Gg  to  G*e.  Note  that 
all  such  transitions  are  guaranteed  to  be  enabled  in  Gg 
(see  line  10  in  Algorithm  3).  Therefore,  given  v*=ev’1’, 
it  follows  that  S*  c  S^ .  That  is,  G*(0 )  is  unique  for 
all  9  e  (0, 1)  in  the  sense  that  the  configured  plant  is 
maximally  permissive  among  all  other  configurations 
that  yield  the  same  optimal  measure  v*(9).  □ 


4.1  Optimal  control  of  non-terminating  plants 

This  section  presents  the  optimal  supervision  problem 
for  non-terminating  plants  (i.e.,  with  termination  prob¬ 
ability  0=0  at  each  state)  having  the  structure 
G(0)  =  ( Q ,  E,  8,  P,  x,  (t?)  and  the  corresponding  stochas¬ 
tic  transition  probability  matrix  is  P.  The  rationale 
for  working  on  a  terminating  plant,  instead  of  the 
non-terminating  plant  is  explained  below. 

By  maximizing  the  measure  v(9)  for  a  given  9  e  (0, 1), 
an  optimal  control  law  can  be  derived  based  on  the  state 
transition  cost  matrix  11(0)  =  (1  —  0)P  of  the  supervised 
plant  language  and  the  originally  assigned  ^-vector. 
Such  an  optimal  control  law  is  sought  to  be  0-indepen- 
dent  in  the  sense  of  having  the  same  disabling  set 
S  Q  for  a  given  range  of  0,  where  0  might  be  restricted 
to  be  not  too  far  away  from  0+.  On  the  other  hand,  from 
the  perspective  of  numerical  stability  and  accuracy  in 
computation  of  v(0)  (see  Definition  8),  it  is  desirable  to 
have  a  relatively  large  positive  value  of  0.  The  results 
derived  in  this  section  serve  toward  establishing  upper 
bounds  on  0  for  which  the  optimal  control  law  should 
be  0-independent  and  the  associated  computation  is 
numerically  stable.  The  main  objective  is  summarized 
below. 

A  uniform  non-zero  probability  of  termination  d„  e  (0, 1)  is  to 

be  computed  such  that  the  terminating  plant  G(6.)  and  the 
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non-terminating  plant  G(0)  shall  have  the  same  the  disabling  set 
O’  c  V.  However,  in  general,  their  measures  could  be  different, 
i.e.,  v(6».)  #E  v(0). 

Proposition  7:  Let  (1  —  0)P  and  v(0 )  be  the  state  transi¬ 
tion  cost  matrix  and  the  measure  of  the  plant 
G(9)  =  (2,  £,<5,(1  ~  0)P,  1,  n  Then,  for  all  qu  qj  e  Q, 
there  exists  0*.  e  (0, 1]  such  that  V0g(O,0*),  the  sign 
of  (vi(9)  —  vj(9 ))  is  fixed  (i.e.,  positive,  negative  or 
zero);  and  0t  can  be  computed  as  an  explicit  function 
of  the  stochastic  matrix  P  and  state  characteristic 
vector  /. 

Proof:  Let  yfO)  =  u,(0)  —  V/(0)  V0  e  (0, 1),  which  is  a 

smooth  function  of  9,  and  yf  0)  =  lim0^o  +  y,y(0)-  The 

proof  is  based  on  the  following  two  cases. 

Case  1:  No  sign  change  of  y,y(0)  in  (0, 1)  =>■  9*j  —  1. 
This  includes:  ytf 0)  =  0  and  {dkyy(9)/d9k)\0=0  =  0  for  all 
k>  0  because  yff  )  =  0  V0  e  (0, 1)  by  Proposition  A. 3. 

Case  2:  y,y(0)  changes  sign  in  (0, 1);  and 

{dryij(9)/d9')\g=0  —  kg  f  0  for  some  integer  r>  0. 

If  r  =  0,  there  exists  t[  e  (0, 1)  such  that  yy( r\)  —  0  for 
the  first  time.  If  /■  >  0,  it  is  possible  that  y,y(0)  =  0. 
Then,  as  9  is  increased  from  zero,  y,y(0)  becomes  non¬ 
zero  and  there  exists  t\  e  (0, 1)  such  that  y;/(Ti )  —  0 
again.  Smoothness  of  y,j(9)  necessitates  that 

(dryy(0)/d0r)\g=o>  —  0  for  some  0t.  €  (0,  ti).  Then,  it 
follows  from  the  Mean  value  Theorem  that  there  exists 
r2  e  (0, 9*1)  such  that 

9r+V#)i  = 

30'+ 1  le=n  9*j 

for  the  given  r  >  0,  Proposition  A. 2,  triangular  inequal¬ 
ity,  and  the  relation  yfff)  —  v,(0)  —  Vj(9 )  yield 


(3' v,<0)/90')|e=o  -  (3'  p/(0)/30'+i)  0=o 

(r+  1)1  2'+3 

infant) 

[n  -p  +  ot.f\  1 

oo/ 

r+ 1 

|{[]  -  P + -A~'  x  -  *xh -t[D-P+-^]-'x- *x)j\  jr  n 

8x(w/^||[0-P  +  <*?r1||J  ’ 

{p-p+.:*r1[l]-[ll-p+.*rTX}i 

-{[o-p+.^r^i-p-p+^T/} 

i  iTT  -i-i  1 1  ’  if  r  >  0 

2'-3(inWo||[D  —  P  +  a#\  ||  J 


Remark  3:  For  a  non- terminating  plant  G(0)  = 
(Q,  E,  <5 ,  P,  x),  let  0*  =  mirpy  9*.  Then,  the  plant  config¬ 
uration  obtained  by  applying  a  single  iteration  of 
Algorithm  3  to  the  0-parameterized  plant  G(0)  — 
(Q,  £,  8 ,  (1  —  0)P,  x, ^)  is  identical  for  all  9  <  0*. 

The  procedure  of  computing  0*  is  summarized  as 
Algorithm  4. 

Proposition  8:  Complexity  of  computing  a  positive 
bound  for  0*  is  O(if)  where  n  is  the  number  of 
plant  states. 


Algorithm  4:  Computation  of  the  Bound  0* 


input  :  P,  x 
output:  0* 

1  begin 


7 

8 
9 

10 
11 
12 

13 

14 

15 

16 

17 

18 

19 

20 
21 
22 

23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 

34 

35  end 


Set  0*  =  1; 

Set  Ocurr  =  0; 
Compute  3P  ; 
*/ 

Compute  Mo  = 
Compute  Mi  = 


/*  See  Algorithm  6  in  Appendix  A 


I-P+, 


I- 


[it  -  p + 


Compute  Mi  =  inf  ^  ||I«F  +  aSP J  ||  ; 

for  j  =  1  to  n  do 
for  i  =  1  to  n  do 

if  i0*x)t  -  (t^x),  +  0  then 

|  9 cun  =  gjjjj  |  ( ^X)i  ~  {3*X) j 

else 

for  r  =  0  to  n  do 

if  (Mo*);  *  (M0*);  then 
Break; 
else 

if  (M0M\x);  *  (m0M!|*)  .  then 
|  Break; 
endif 
endif 
endfor 
if  r  ==  0  then 

|  n  {M0-3*)x,-(M0-&)Xi\ . 

&curr  — 


8M2 


else 


Ocurr  — 

else 

|  0clirr  =  1 

endif 
endif 
endif 

Or,  =  min(0*,  0curr )  ; 

endfor 
endfor 


if  r  >  0  AND  r  <n  then 


2r+3M2 


(23) 

□ 


1282 


I.  Chattopadhyay  and  A.  Ray 


Proof:  Referring  to  Algorithm  4,  the  part  within  the 
nested  For  loops  (lines  10  to  32)  is  executed  at  most  nr 
times  and  each  iteration  involves  only  single-iteration 
scalar  operations.  Thus  the  computational  complexity 
of  this  part  is  of  the  order  of  0(n2).  Lines  5  and  6  involve 
inversion  of  n  x  n  dimensional  non-singular  matrices 
and  hence  the  complexity  of  execution  is  of  the  order 
of  0(n 3).  Proposition  G  (see  Appendix)  guarantees 
that  the  complexity  of  computing  SP  is,  in  general, 
of  the  order  of  0(n 3).  Line  7,  which  computes 
Mi  —  infa^o  ||[D  —  P  +  ctQ.fy1  Hqq,  is  a  search  problem. 
However,  since  Mi  appears  only  in  the  denominator  of 
the  expressions  for  0curr,  it  follows  that,  if  for  some 
a  —  a0  /  0  and  by  using 


Mi 


[D  -  P  + 


OO 


(24) 


it  is  possible  to  obtain  a  positive  lower  bound  of  0* 
in  Algorithm  4.  Since  the  computation  of 
||[D  —  P  +  o'o-^]”1||00  is  of  the  order  of  0(«3)  due  to  the 
matrix  inversion,  it  is  concluded  that  a  positive  lower 
bound  of  0,  can  be  computed  with  a  complexity 
of  0(«3).  □ 

Remark  4:  It  is  shown  in  (Chattopadhyay  and  Ray 
2006a)  that  for  any  stochastic  matrix  P 


[D-P  +  a,^]  1  =  [D  —  P  -F  1 

1  —  QfN 


9  Va^O 


.([D  — P+./j  +  (25) 


Using  M2  —  ||  [D  —  P  +  rCp'lloo  instead  of 
Mi  —  infa^o  l|[D  -  P  +  a.fyX  lloo  (i-e.,  using  a  —  1)  in 
Algorithm  4  yields  a  value  which  satisfies  the  require¬ 
ment  stated  in  Remark  3  and  therefore  qualifies  as  0*. 
Thus,  the  major  advantage  of  this  approximation  is 
having  significantly  smaller  computational  complexity 
because  the  search  involved  in  computing  the  infimum 
is  avoided  at  the  cost  of  using  a  smaller  value  of  0*, 
which  may  make  subsequent  computation  of  measure 
slightly  more  difficult  due  to  possible  ill-conditioning 
(see  Definition  8). 

On  account  of  Proposition  7  and  Remark  3,  Algorithm  3 
is  modified  to  solve  the  optimal  supervision  problem 
for  non-terminating  plants  and  the  modified  version  is 
formally  presented  in  Algorithm  5. 


Algorithm  5:  Computation  of  Optimal  Supervi¬ 
sor 


input  :  P,  X,  S? 

output:  Optimal  set  of  disabled  transitions 

begin 

Set  =  0  ;  /*  Initial  disabling  set 

Set  rf°J  =  n  :  /*  Initial  event  prob.  matrix 

Set  0™  =  0.99; 

Set  k  -  1  : 

Set  Terminate  =  false  ; 
while  (Terminate  ==  falsej  do 


Compute  6 


Ik]. 


/ *  Algorithm  4 


9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
21 
22 

23 

24 

25  end 


Set  ffM  =  d-iLri*-1!; 
t-e'C’1 

Compute  ; 

for  j  =  1  to  n  do 
for  i  =  1  to  n  do 

Disable  all  controllable  transitions  <7, 

such  that  <  v!®  ; 

I  1 

Enable  all  controllable  transitions  q-, 

such  that  v1.^  >  vP1  ; 

I  ~  ‘ 

endfor 

endfor 

Collect  all  disabled  transitions  in  54® ; 
if  ==  then 

|  Terminate  =  true  ; 
else 

|  k  =  k  +  1  ; 

endif 

endw 

c%*  -  gdi|  .  /*  Optimal  disabling 


set 


*/ 

*/ 


*/ 


*/ 


Proposition  9  (effectiveness):  Algorithm  5  is  an  effec¬ 
tive  procedure  (Hopcroft  et  al.  2001),  i.e.,  it  is  guaran¬ 
teed  to  terminate. 

Proof:  Comparison  of  Algorithm  3  and  Algorithm  5 
reveals  that  while  the  former  assumes  a  fixed  probability 
of  termination  0  at  each  state,  the  latter  modifies 
this  parameter,  denoted  as  0^,  at  each  iteration  k.  Let 
dmin  =min(0W,0pl)  and  let  S>[1](0min)  and  2>[2](0min)  be 
sets  of  disabled  transition  at  the  first  and  second  itera¬ 
tions,  respectively,  for  the  terminating  plant  G(0mjn). 
Similarly,  for  the  non-terminating  plant  G(0),  let 
3>^(0)  and  S^1(0)  be  the  sets  of  disabled  transitions 
at  the  first  and  second  iterations,  respectively.  It 
follows  from  Remark  3  that  ®^1(0)  =  @^(0m in)  and 

®[2](0)  =  S>[2](0min). 

Extending  the  above  argument  by  induction  based 
on  k  iterations  of  Algorithm  5  and  denoting 
0min  =  min(0jd, . . . ,  0^1),  an  application  of  Algorithm  3 
on  a  terminating  plant  G(0min)  yields 

#-](0)=  #](0min)  Vre  k}. 
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Proposition  4  states  that,  for  an  arbitrary  plant, 
Algorithm  3  is  guaranteed  to  terminate  within  finitely 
many  iterations.  Hence,  Algorithm  5  is  an  effective 
procedure.  □ 

Next,  it  is  shown  that  the  plant  configuration  obtained 
by  Algorithm  5  is  optimal  in  the  sense  of  Definition  13. 

Proposition  10  (optimality):  For  a  non-terminating 
plant  G(0)  =  (Q,  E,<5,  P,  %,  W)),  the  supervision  policy 
computed  by  Algorithm  5  is  optimal  in  the  sense  of 
Definition  13. 

Proof:  Let  the  set  of  disabled  transitions  computed  at 
the  kth  iteration  Algorithm  5  be  denoted  by  and 
the  termination  probability  be  denoted  by  (fifi. 
Let  the  set  of  disabled  transitions  at  the  convergence 
of  Algorithm  5  be  Let  (9mi„  =  minr6{i....,£) 

(0[1],...,0M)>O. 

Let  G(0m  jn)  be  a  terminating  plant  with 
IKflmin)  =  (1  —  ftnin)P.  It  follows  from  the  proof  of 
Proposition  9  that  applications  of  Algorithm  3  to 
G(0min)  and  Algorithm  5  to  G(0)  yield  the  same  set  3 
of  disabled  controllable  events  although  the  optimal 
measures,  being  0-dependent  would  be  different,  i.e., 
v(0min)  /e  v(0). 

Proposition  5  implies  that  the  optimal  disabling  set 
for  a  plant  G(0)  generates  the  the  same  set  of  disabled 
controllable  transitions  for  all  0  <  6  =  6m in-  Because  of 
continuity  of  v(0)  with  respect  to  0,  it  is  argued  that 
G*( 0)  is  optimal  in  the  sense  of  Definition  13,  i.e., 
v*  =e  v1',  where  G^(0)  is  obtained  by  arbitrarily  disabling 
controllable  transitions  in  G.  This  completes  the 
proof.  □ 

Next  it  is  shown  that  the  supervision  policy  computed  by 
Algorithm  5  is  unique  in  the  same  sense  as 
Proposition  6. 

Proposition  11  (Uniqueness):  Let  G(0)  be  an  unsuper¬ 
vised  non-terminating  plant  and  G*(0)  be  the  supervised 
plant  configured  by  Algorithm  5.  Then,  G*  is  unique  in 
the  sense  that  it  is  maximally  permissive  among  super¬ 
vised  plants  that  yield  optimal  performance  based  on 
0- neighbours  G(0)  of  G(0)  (see  Definition  11)  for  all 
0  e  (0, 0*),  where  0,  is  computed  by  Algorithm  4. 
Equivalently,  if  G1’(0)  is  an  arbitrarily  supervised  plant, 
then  the  following  condition  holds: 

(v*(0)  vf(0))  /\  (V  c  #)  V  (V*(0)  ^E  Vf(0)))  , 

where  v  and  3  denote  respective  language  measures  and 
sets  of  disabled  transitions. 


Proof:  It  follows  from  Proposition  10  that 
v*(0)  =e  v^(0).  It  also  follows  from  Proposition  that 
v*(0)  =^E  v^(0)  for  0  e  (0,0,).  If  v*(0)  =E  v^(0),  then  G*(0) 
and  Gfd)  are  both  optimal  supervised  configurations 
of  the  unsupervised  terminating  plant  G(0).  It  follows 
from  Proposition  6  that  3*  c  3^;  otherwise 
V*(0)/EVf(0).  □ 

Proposition  12:  Computational  complexity  of 
Algorithm  5  is  of  the  same  order  as  that  of  Algorithm  3. 

Proof:  Algorithm  5  computes  0*  in  each  iteration  and 
complexity  of  this  computation  is  0(«3),  where  n  is  the 
number  of  states  in  the  plant  (see  Proposition  8).  Each 
iteration  of  both  Algorithm  3  and  Algorithm  5  involves 
computation  of  the  measure  vector  v,  whose  complexity 
is  also  0(«3)  because  of  n  x  n  matrix  inversion.  Hence, 
computational  complexity  of  each  iteration  is  0(n3) 
for  both  Algorithm  3  and  Algorithm  5.  Finally,  the 
argument  presented  in  Proposition  9  implies  that 
the  number  of  iterations  in  Algorithm  5  is  of  the 
same  order  as  that  in  Algorithm  3.  This  completes 
the  proof.  □ 

4.2  Testing  of  computational  complexity 

Proposition  4  shows  that  Algorithm  3  is  an  effective 
procedure  (Hopcroft  et  al.  2001),  i.e.,  the  solution  is 
guaranteed  to  converge  in  a  finite  number  of  iterations. 
Extensive  simulation  suggests  that  the  the  maximum 
number  of  iterations  for  Algorithm  3  is  actually  of  poly¬ 
nomial  order  in  n,  where  n  is  the  number  of  states  in  the 
unsupervised  plant.  The  result  is  illustrated  in  figure  6, 
where  the  maximum  number  of  required  iterations 
/max  is  plotted  against  number,  n,  of  plant  states.  For 
each  n,  10,000  simulation  runs  were  conducted  for 
synthesis  of  optimal  plant  configuration  with  randomly 
generated  entries  in  the  pair  ((1  —  0)P,  /);  and  7max 
was  chosen  to  be  the  maximum  number  of  iterations 
required  by  Algorithm  3  to  converge;  this  is  the  most 
conservative  case.  The  plot  in  figure  1  shows  a  distinct 
sub-linear  variation.  The  following  conjecture  is  made 
based  on  these  observations. 

Conjecture  1  (polynomial  convergence):  Given  a  termi¬ 
nating  plant  G(6)  with  a  uniform  non-zero  probability  of 
termination  0  at  each  of  the  n  plant  states, 

1.  Algorithm  3  converges  in  at  most  0{n)  iterations. 

2.  Computational  complexity  of  Algorithm  3  is  bounded 
by  0(n4). 

Statement  2  in  Conjecture  1  follows  from  Statement  1 
and  the  following  facts:  Each  iteration  has  complexity 
of  G(/73)  due  to  matrix  inversion  in  the  computation  of 
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the  language  measure  vector,  and  matrix  inversion  has 
complexity  of  0(«3)).  Combination  of  Conjecture  1 
and  Proposition  12  implies  that  Algorithm  5  converges 
in  O(n)  iterations  and  that  complexity  of  the  algorithm 
is  0(n4).  Similar  to  the  procedure,  described  above  for 
Algorithm  3,  10,000  random  simulation  runs  for  each 
n  were  conducted  for  testing  Algorithm  5.  Figure  2 
shows  the  plot  of  average  number  of  iterations  required 
to  converge  at  each  value  of  n  in  contrast  to  figure  1, 
where  the  maximum  number  of  iterations  is  potted.  As 
expected,  the  plot  of  figure  2  is  also  sub-linear. 

5.  Optimal  control  of  three  processor  message  decoding 

This  section  presents  the  design  of  a  discrete-event 
(controllable)  supervisor  for  a  multiprocessor  message 


Figure  3.  Arrangement  of  the  processor  links. 


decoding  system,  described  in  an  earlier 
publication  (Ray  et  al.  2004).  The  optimal  supervisory 
algorithm  has  been  synthesized  based  on  the  algorithms 
presented  in  earlier  sections. 

Figure  3  depicts  the  arrangement  of  the  message 
decoding  system,  where  each  of  the  three  processors, 
Pi,  p2  and  p2,  receives  encoded  messages  that  are  to 
be  decoded.  The  processor  p2  normally  receives  the 
most  important  messages,  and  px  receives  the  least 
important  messages.  There  is  a  server  between  each 
pair  of  processors — si  between  px  and  p2;  s2  between 
p2  and  py  and  s3  between  p3  and  px.  Each  server  is 
connected  to  each  of  its  two  adjacent  processors  by 
a  link — the  server  Sj  is  connected  to  the  adjacent  pro¬ 
cessors  pi  and  pk  through  the  links  L,j  and  L y,  respec¬ 
tively.  Out  of  these  six  links,  each  of  the  three  links, 
L]  i ,  L 12,  and  L2\ ,  is  equipped  with  a  switch  to  disable 
the  respective  connection  whenever  it  is  necessary; 
each  of  the  remaining  three  links,  L22,  L22,  and  L33, 
always  remain  connected.  Each  server  st  is  equipped 
with  a  decoding  key  kt  that,  at  any  given  time,  can 
only  be  accessed  by  only  one  of  the  two  processors, 
adjacent  to  the  server,  through  the  link  connecting 
the  processor  and  the  server.  In  order  to  decode  the 
message,  the  processor  holds  the  information  on 
both  keys  of  the  servers  next  to  it,  one  at  a  time. 
After  decoding,  the  processor  simultaneously  releases 
both  keys  so  that  other  processors  may  obtain 
access  to  them. 

Figure  4  depicts  the  unsupervised  plant  model  of  the 
decoding  system  as  a  finite  state  automaton,  where 
state  1  is  the  initial  state.  The  event  py  indicates  that 
processor  /?,  has  accessed  the  key  kf,  and  the  event  f 
indicates  that  the  processor  pt  has  finished  decoding 
and  (simultaneously)  released  both  keys  in  its  possession 
upon  completion  of  decoding.  The  events  f 
are  uncontrollable  because,  after  the  decoding  is 
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initiated,  there  is  no  control  on  when  a  processor  finishes 
decoding. 

Table  1  lists  the  event  cost  matrix  II.  Two  different 
control  specifications  are  investigated.  The  first  set  of 
specifications,  which  emphasizes  avoiding  deadlock,  is 
represented  by  the  7  vector  in  the  first  column  of 
table  2.  The  second  set  of  specifications,  which  focuses 
on  increasing  the  throughput  of  processor  1,  is  repre¬ 
sented  by  the  7  vector  in  the  second  column  of  table  2. 
The  positive  elements  of  the  7  vector  are  assigned  to 
the  states  8  to  16  that  represent  successful  decoding  of 
each  processor.  The  7  values  of  the  deadlock  states  26 
and  27,  where  each  processor  holds  exactly  one  key 
and  hence  no  processor  releases  its  key,  are  assigned 
negative  values.  The  remaining  states  are  non-marked 
and  are  assigned  zero  weights. 

Algorithm  5  is  applied  to  obtain  the  sequence  of 
measure  vectors  for  the  two  control  specifications. 
The  results  of  successive  iterations,  enumerating  the 
renormalized  measure  vectors,  are  presented  in  Table  3 
and  4  respectively.  The  last  column  in  each  table  is 
the  optimal  renormalized  measure  vector.  The 
optimization  requires  7  iterations  in  Case  1  and  5 
iterations  in  Case  2. 

The  optimal  configurations  for  the  plant  obtained 
under  Algorithm  5  are  depicted  in  figures  5  and  6  respec¬ 
tively.  For  supervisor  policy  1,  the  controlled  plant  is 
not  trim  and,  for  supervisor  policy  2,  there  are  discon¬ 
nected  states  in  the  controlled  model.  This  is  interpreted 
as  the  supervisor  successfully  preventing  the  plant 
from  visiting  these  states.  The  critical  values  for  the 
termination  probability  d*  computed  by  the  optimiza¬ 
tion  algorithm  for  each  control  specification  is  shown 
in  figure  7. 


Table  1.  Event  occurrence  probabilities  for 
processor  models. 


p\\ 

pi  3 

p21 

p22 

p32 

p33 

/I 

/2 

/3 

0.16 

0.04 

0.16 

0.16 

0.16 

0.32 

0.00 

0.00 

0.00 

0.00 

0.16 

0.00 

0.26 

0.26 

0.32 

0.00 

0.00 

0.00 

0.37 

0.00 

0.21 

0.21 

0.21 

0.00 

0.00 

0.00 

0.00 

0.32 

0.11 

0.26 

0.00 

0.00 

0.32 

0.00 

0.00 

0.00 

0.00 

0.11 

0.00 

0.28 

0.28 

0.33 

0.00 

0.00 

0.00 

0.25 

0.00 

0.25 

0.25 

0.25 

0.00 

0.00 

0.00 

0.00 

0.28 

0.11 

0.28 

0.00 

0.00 

0.33 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.39 

0.39 

0.00 

0.22 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

1.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

1.00 

0.00 

0.00 

0.00 

0.14 

0.00 

0.00 

0.00 

0.43 

0.00 

0.43 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

1.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

1.00 

0.00 

0.33 

0.00 

0.33 

0.00 

0.00 

0.00 

0.00 

0.00 

0.34 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

1.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

1.00 

0.00 

0.00 

0.00 

0.50 

0.50 

0.00 

0.00 

0.00 

0.00 

0.00 

1.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.25 

0.00 

0.00 

0.00 

0.75 

0.00 

0.00 

0.00 

0.50 

0.00 

0.50 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.50 

0.00 

0.50 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.50 

0.50 

0.00 

0.00 

0.00 

0.00 

0.50 

0.00 

0.50 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.50 

0.50 

0.00 

0.00 

0.00 

0.00 

0.00 

0.25 

0.00 

0.00 

0.00 

0.75 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.33 

0.33 

0.34 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.33 

0.33 

0.34 

Table  2. 

Vectors  for  control  specifications. 

Case  1 

Case  2 

0.000 

0.010 

0.000 

0.000 

1.000 

0.000 

0.000 

0.020 

0.000 

0.000 

0.020 

0.000 

0.000 

0.020 

0.000 

0.000 

0.020 

0.000 

0.000 

0.020 

0.000 

0.000 

0.020 

0.000 

0.000 

0.040 

0.000 

0.000 

0.040 

0.000 

0.000 

0.040 

0.000 

0.000 

0.040 

0.000 

0.000 

0.040 

0.000 

0.000 

0.040 

0.000 

0.010 

0.000 

-1.000 

1.000 

0.000 

-0.200 

0.010 

0.000 

-1.000 

1.000 

0.000 

-0.200 

Next  the  stable  probability  distributions  of  the  plant 
states  are  compared  for  the  following  three  cases: 

•  Open-loop  or  unsupervised  plant 

•  Plant  with  the  optimal  supervision  policy  for 

specification  1 

•  Plant  with  the  optimal  supervision  policy  for 

specification  2 

The  distributions  are  obtained  by  considering  the  first 
row  of  the  matrix  based  on  the  measure  Vi 
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Table  3.  Iteration  vectors  for  multi-processor  model:  case  1.  Table  4.  Iteration  vectors  for  multi-processor  model:  case  2. 


Itr  1 

Itr  2 

Itr  3 

Itr  4 

Itr  5 

Itr  6 

Itr  7 

Itr  1 

Itr  2 

Itr  3 

Itr  4 

Itr  5 

-0.0616 

0.0006 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2076 

0.2879 

0.3245 

0.3245 

-0.0616 

0.0001 

0.0063 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2074 

0.2880 

0.3245 

0.3245 

-0.0616 

0.0003 

0.0055 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2101 

0.2879 

0.3245 

0.3245 

-0.0616 

0.0007 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2167 

0.2876 

0.3232 

0.3245 

-0.0616 

0.0011 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2109 

0.2878 

0.3236 

0.3245 

-0.0616 

0.0012 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2084 

0.2882 

0.3245 

0.3245 

-0.0616 

0.0000 

0.0093 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2059 

0.2878 

0.3245 

0.3245 

-0.0616 

0.0002 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2090 

0.2879 

0.3245 

0.3245 

-0.0616 

0.0000 

0.0093 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2059 

0.2878 

0.3245 

0.3245 

-0.0616 

0.0007 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2175 

0.2875 

0.3230 

0.3245 

-0.0616 

0.0007 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2089 

0.2879 

0.3245 

0.3245 

-0.0616 

0.0003 

0.0055 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2105 

0.2879 

0.3245 

0.3245 

-0.0616 

0.0013 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2086 

0.2882 

0.3245 

0.3245 

-0.0616 

0.0010 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2078 

0.2879 

0.3245 

0.3245 

-0.0616 

0.0011 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2114 

0.2878 

0.3235 

0.3245 

-0.0616 

0.0001 

0.0063 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2076 

0.2880 

0.3245 

0.3245 

-0.0616 

0.0001 

0.0000 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2080 

0.2880 

0.3245 

0.3245 

-0.0616 

0.0000 

0.0093 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2059 

0.2878 

0.3245 

0.3245 

-0.0616 

0.0009 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2216 

0.2872 

0.3241 

0.3245 

-0.0616 

0.0000 

0.0000 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2059 

0.2878 

0.3245 

0.3245 

-0.0616 

0.0003 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2147 

0.2879 

0.3245 

0.3245 

-0.0616 

0.0003 

0.0000 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2116 

0.2879 

0.3245 

0.3245 

-0.0616 

0.0014 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2084 

0.2879 

0.3245 

0.3245 

-0.0616 

0.0012 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2105 

0.2878 

0.3232 

0.3245 

-0.0616 

0.0012 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2110 

0.2878 

0.3236 

0.3245 

-0.0616 

0.0007 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2077 

0.2879 

0.3245 

0.3245 

-0.0616 

0.0007 

0.0110 

0.0124 

0.0124 

0.0143 

0.0143 

0.0598 

0.2077 

0.2879 

0.3245 

0.3245 

corresponding  to  state  1  which  is  the  initial  state  in  both 
cases.  If  the  stochastic  matrix  P  is  primitive  (i.e.,  irredu¬ 
cible  and  acyclic),  then  all  rows  of  SP  would  be  identical. 
However,  primitiveness  of  P  is  not  guaranteed  even  if 
the  unsupervised  plant  model  have  this  property  because 
any  subsequent  event  disabling  may  cause  loss  of 
reducibility  or  acyclic  properties. 

The  results  on  evolution  of  the  distribution  are  plotted 
in  figure  8.  While  the  unsupervised  plant  has  a  finite 
probability  of  reaching  the  deadlock  states  26  and  27, 
the  optimal  supervisors  in  both  cases  successfully 
prevent  occurrence  of  deadlock  in  the  sense  that  the 
stable  occupation  probabilities  for  states  26  and  27  are 
zero  for  each  supervisor.  However,  supervisor  2 
increases  the  throughput  of  processor  1  as  seen  from 
the  increased  probability  of  occupying  states  1  and  2. 

6.  Summary,  conclusions,  and  recommendations  for 
future  work 

This  paper  presents  the  theory,  formulation,  and 
validation  of  optimal  supervisory  control  policies  for 
dynamical  systems,  modelled  as  probabilistic  finite 


Figure  5.  Optimal  plant  configuration  for  specification  1. 


state  automata.  The  procedure  for  synthesis  of  the  opti¬ 
mal  control  policy  relies  on  a  (renormalized)  signed 
real  measure  of  regular  languages  (Chattopadhyay  and 
Ray  2006a)  to  construct  the  performance  index.  The 
language  measure  is  based  on  the  state  transition 
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Figure  7.  Computed  for  each  iteration. 


probability  matrix  of  the  underlying  finite-state  Markov 
chain  model  of  the  process  and  a  characteristic  vector  of 
state  weights,  which  serves  as  the  control  specification. 

The  main  contribution  of  the  paper  is  reformulation 
of  the  optimal  supervisor  synthesis  algorithm  (Ray 
et  al.  2004,  2005)  for  probabilistic  finite  state  plant 
models  in  terms  of  the  renormalized  measure  and 
extension  of  the  technique  to  general  non-terminating 
probabilistic  models.  Specifically,  the  work  reported  in 
this  paper  removes  a  fundamental  restriction  of  earlier 
analysis  (Ray  et  al.  2004,  Ray  2005),  namely,  each  row 
sum  of  the  state  transition  cost  matrix  II  being  strictly 
less  than  one,  instead  of  being  exactly  equal  to  one. 
The  novel  concept  of  language-based  control  synthesis, 
presented  in  this  paper,  allows  quantification  of  plant 


Figure  8.  Stable  state  probability:  unsupervised  and  super¬ 
vised  plants. 


performance  instead  of  solely  relying  on  its  qualitative 
performance  (e.g.,  permissiveness),  which  is  the  current 
state  of  the  art  for  discrete  event  supervisory 
control  (Ramadge  and  Wonham  1987,  Cassandras  and 
Lafortune  1999). 

The  following  conclusion  is  drawn  in  view  of  using  the 
language  measure  for  construction  of  the  performance 
index  for  deriving  an  optimal  control  policy.  Like  any 
other  optimization  procedure,  it  is  possible  to  choose 
different  performance  indices  to  arrive  at  different 
optimal  policies  for  discrete  event  supervisory  control. 
Nevertheless,  usage  of  the  language  measure  provides 
a  systematic  procedure  for  precise  comparative  evalua¬ 
tion  of  different  supervisors  so  that  the  optimal  control 
policy(ies)  can  be  unambiguously  identified.  These 
theoretical  results  also  lay  the  foundation  for  extension 
of  the  language-measure-theoretic  framework  to  plant 
modelling  and  control,  where  all  events  may  not  be 
observable  at  the  supervisory  level. 

The  paper  provides  details  of  the  algorithms  that  are 
required  for  synthesis  of  the  optimal  supervisory  control 
policy.  These  algorithms  are  executable  in  real  time  on 
commercially  available  platforms.  Computational  com¬ 
plexity  of  the  presented  algorithms  is  polynomial  in 
the  number  of  plant  model  states.  The  concepts  are 
elucidated  with  simple  examples  and  a  relevant  engineer¬ 
ing  example.  As  such  it  is  straight-forward  to  develop 
real-time  software  codes  in  standard  languages,  based 
on  the  algorithms  provided  in  this  paper. 

There  are  several  issues  that  need  to  be  addressed  for 
implementation  of  the  theory  of  discrete-event  supervi¬ 
sory  control  in  an  operating  plant.  For  example,  the 
events  must  be  generated  in  real  time,  based  on  physical 
measurements,  to  provide  the  supervisor  with  the 
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current  information  on  the  plant;  this  is  beyond  what  is 
done  off-line  for  construction  of  the  plant  model  and 
control  synthesis.  Similarly,  the  event  disabling/enabling 
decisions  of  the  supervisor  must  be  translated  in  real 
time  as  appropriate  actions  to  control  the  plant. 


6.1  Recommendations  for  future  research 

Synthesis  of  supervisory  control  systems  may  become  a 
significant  challenge  if  some  of  the  events  are  delayed, 
intermittent,  or  not  observable  at  all,  possibly  due  to 
sensor  faults  or  malfunctions  in  network  communication 
links.  In  that  case,  the  control  algorithms  may  turn  out 
to  be  computationally  very  complex  because  of 
delayed  or  lost  information  on  the  plant  dynamics. 
Future  work  in  this  direction  should  involve  research 
on  construction  of  language  measures  under  partial 
observation  (Chattopadhyay  and  Ray  2006b)  and 
associated  synthesis  of  optimal  control  policies  to  miti¬ 
gate  the  detrimental  effects  of  loss  of  observability. 
The  latter  research  could  be  an  extension  of  the  work 
on  optimal  control  under  full  observation,  reported  in 
this  paper. 

It  would  be  a  challenging  task  to  extend  the  concept 
of  (regular)  language  measure  for  languages  higher  up 
in  the  Chomsky  hierarchy  (Flopcroft  et  al.  2001)  such 
as  context-free  and  context-sensitive  languages.  This 
extension  would  lead  to  controller  synthesis  when  the 
plant  dynamics  is  modelled  by  non-regular  languages 
such  as  the  Petri  net  (Cassandras  and  Lafortune  1999, 
Murata  1989).  The  research  thrust  should  focus 
on  retaining  the  polynomial  order  of  computational 
complexity. 

Another  critical  issue  is  how  to  extend  the  language 
measure  for  timed  automaton,  especially  if  the  events 
are  observed  with  varying  delays  at  dilferent  states. 
Another  research  topic  that  may  also  be  worth  investi¬ 
gating  is:  how  to  extend  the  GF( 2)  field,  over  which 
the  vector  space  of  languages  is  defined  (Ray  2005),  to 
richer  fields  like  the  set  of  real  numbers. 

Areas  of  future  research  also  include  applications  of 
the  language  measure  in  anomaly  detection,  model  iden¬ 
tification,  model  order  reduction,  and  analysis  and 
synthesis  of  interfaces  between  the  continuously-varying 
and  discrete-event  spaces  in  the  language -measure 
setting.  Future  research  for  advancement  of  the  theory 
of  optimal  supervisory  control  for  discrete  event  systems 
include  the  following  areas: 

•  Robustness  of  the  control  policy  relative  to  unstruc¬ 
tured  and  structured  uncertainties  in  the  plant  model 
including  variations  in  the  language  measure 
parameters  (Lagoa  et  al.  2005) 

•  Control  synthesis  under  partial  observation  to  accom¬ 
modate  loss  of  observability  at  the  supervisory  level 


possibly  due  to  sensor  faults  or  communication  link 
failures  (Chattopadhyay  and  Ray  2006b) 

•  Construction  of  grammar-based  measures,  instead  of 
memory-less  state-based  measures  (Chattopadhyay 
and  Ray  2005),  for  non-regular  languages  when 
details  of  transitions  in  plant  dynamics  cannot  be 
captured  by  finitely  many  states 
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Appendix  A:  Derivatives  of  renormalized  measure 

This  appendix  establishes  bounds  on  the  derivatives  of 
the  renormalized  measure  v(0)  for  all  6  e  (0, 1)  and 
computes  the  limits  of  the  derivatives  as  0  ->  0+  as  an 
extension  of  what  was  reported  in  the  previous 
publication  (Chattopadhyay  and  Ray  2006a). 

The  main  result  on  boundedness  of  the  derivatives  of 
v(9)  are  presented  as  propositions.  Specifically,  the 
results  reported  in  Chattopadhyay  and  Ray  (2006a) 
are  combined  as  the  next  two  propositions. 

Proposition  A.l:  Let  4/(0)  =  0[O  —  (1  —  0)P]_1,  where  P 
is  a  («  x  n )  stochastic  matrix  and  n  e  N.  Then, 


(i)  Vfce  N\{1} 

lim  =  -k  lira 

0->O+  d0k  0+ 


a*  - 14/(0) 

36k  -  1 


[P  +  -J/][D  -  P  +  //]-' 


(ii) 


lim 

0->O+ 


1 4<(0) 
99k 


[l-P  +  ./f1  -.9,  ifk—l 

(-l^i-p+^r1 

xfD-tn-P  +  ^r'f-1,  i/JteN\|  1}. 


Proof:  Given  in  Chattopadhyay  and  Ray  (2006a,  §3, 
pp.  1111-1112  as  Corollary  3  and  Corollary  6).  □ 

The  next  proposition  establishes  bounds  on  the  deriva¬ 
tives  of  v(0)  in  an  elementwise  sense  by  computing 
bounds  on  the  induced  sup-norm  of  the  derivatives  of 
4/(0).  Recall  that  x  has  been  defined  to  have  infinity 
norm  equal  to  1. 
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Proposition  A.2 


dkv(9) 

<k\  2k+'(  inf 

[D-P  +  a./l  1 

dOk 

Wo 

OO  \ 

V0e[O,l]. 


Proof:  Given  in  of  Chattopadhyay  and  Ray  (2006a, 
§3,  p.  1113  as  Proposition  5).  □ 

Proposition  A.3:  Denoting  the  /th  element  of  the  fcth 
derivative  of  the  measure  vector  as  (9* v(0)/90/')|;,  it 
follows  that 


Wk  e 


9iv(6>) ,  &v{6) , 

ddk  I'"-  dek  o 


•  VO  e  [0,  l],v(0)l.  =  v(i 


Algorithm  6:  Computation  of  3? 


input  :  P.  Desired  precision  eps 

output: 

1  begin 

2  Set  QM  =  0  ; 

3  Set  A  =  I;  /* 

4  Set.  k  =  1 ; 

5  Set  Terminate  =  false  ; 

6  while  (Terminate  ==  false!  do 

7  Q,irI  =  Q[*"  t  +  1  (A  -  Q[m1): 

8  if  IIQtfcl  -  Q[ifc“i:l]|  <  eps  then 

9  |  Terminate  ==  true  ; 

to  else 

1 1  k=k+ 1 ; 

12  A  =  AP: 

13  endif 

14  endw 

15  0  =  Qlk  h 

16  end 


/*  Zero  Matrix 
Indentity  Matrix 


*/ 

*/ 


where  n  is  the  number  of  states  in  the  plant  model. 
Proof:  First  it  is  noted  that 

OO 

v(6)  =  eJ2(\-  0)kPkX  V0  e  (0, 1] 
k=0 
oo 

=  0^nA(0)zV0e(O,l].  (26) 

k=  0 

Since  11(0)  is  a  matrix  of  dimension  n  x  n,  it  follows 
from  the  Cayley-Hamilton  Theorem  (Bapat  and 
Raghavan  1997)  that  integral  powers  of  11(0)  can  be 
expressed  as  polynomials  of  degree  n—  1  as  follows: 

72—1 

Vr  e  N,  n'  (0)  =  Y  CkVLk(9)  with  ck  e  C.  (27) 

k= 0 

Since  each  term  in  the  summation  on  the  left  hand  side 
of  equation  (26)  is  a  polynomial  in  0  of  degree  n  —  1 ,  it 
follows  that  the  summation  is  also  a  polynomial  in 
degree  n  —  1  (since  the  summation  exists  due  to  the 
sub-stochastic  property  of  11(0)).  Then  it  follows  that 
each  element  of  v(0)  is  a  polynomial  of  degree  n.  The 
result  then  follows  from  continuity.  □ 

Proposition  A.4:  For  any  stochastic  matrix  P  of 
dimension  n  x  n,  the  complexity  of  computing  the  limit¬ 
ing  matrix  3P  is  of  the  order  of  0(n3). 

Proof:  Since  the  limit  lim^oc  ( 1  /k)  P;  —  & 
always  exists,  it  is  possible  to  compute  -f  within  any 
specified  precision  simply  by  computing  the  sum 
F'  followed  by  division  by  k,  for  a  large 
enough  value  of  k.  The  procedure  is  summarized  in 
Algorithm  6. 


Referring  to  Line  7  of  Algorithm  6,  it  is  observed  that 
(fk\  is  a  stochastic  matrix  for  all  k  and  hence  it  follows 
that  the  algorithm  is  guaranteed  to  terminate  in 
(1/eps)  iterations,  independent  of  n.  Each  iteration 
involves  a  single  matrix  multiplication  (P  x  A)  and 
hence  algorithmic  complexity  is  of  the  same  order  as 
multiplication  of  two  n  x  n  matrices,  i.e.,  <  0(«3). 
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The  signed  real  measure  of  regular  languages  has  been  introduced  and  validated  in  recent 
literature  for  quantitative  analysis  of  discrete-event  systems.  This  paper  reports  generalizations 
of  the  language  measure,  which  can  serve  as  performance  indices  for  synthesis  of  optimal 
discrete-event  supervisory  decision  and  control  laws.  These  generalizations  eliminate  a 
user-selectable  parameter  in  the  original  concept  of  language  measure.  The  concepts  are 
illustrated  with  simple  examples. 


1.  Introduction 

In  the  discrete-event  setting,  a  finite-state  automaton 
(FSA)  model  of  a  physical  plant  is  a  generator  of  its 
regular  language,  whose  behaviour  is  constrained  by 
the  supervisor  (or  controller)  to  meet  a  given  specifica¬ 
tion.  A  signed  real  measure  of  regular  languages  has 
been  reported  in  Ray  (2005)  and  Ray  et  al.  (2005)  to 
provide  a  mathematical  framework  for  quantitative 
comparison  of  controlled  sublanguages.  In  this  work, 
each  transition  is  assigned  a  cost,  similar  to  its  probabil¬ 
ity  measure  that  can  be  quantitatively  evaluated  from 
physical  experimentation  or  extensive  simulation  on  a 
test  bed.  Each  state  of  the  FSA  model  is  assigned  a 
signed  real  weight  whose  upper  and  lower  bounds  are 
normalized  to  1  and  —1,  respectively.  The  measure  of 
a  given  event  trace  is  obtained  as  the  product  of  the 
cost  of  transitions  and  the  (normalized)  weight  of  the 
terminating  state.  The  sum  of  the  measures  of  all 
traces  yield  the  language  measure. 

Optimal  control  of  finite  state  automata  has  been 
recently  reported  Ray  et  al.  (2004,  2005)  based  on  the 
total  ordering  induced  by  the  language  measure  as  aug¬ 
mentation  to  the  supervisory  control  theory  of  Ramadge 
and  Wonham  (1987).  This  work  consolidates  the  theory 
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and  applications  of  optimal  supervisory  control  of 
regular  languages,  where  the  performance  index  is 
obtained  by  combining  a  real  signed  measure  of  the 
supervised  plant  language  with  the  cost  of  disabled 
event(s).  Starting  with  the  (regular)  language  of  an  unsu¬ 
pervised  plant  automaton,  the  optimal  control  policy 
makes  a  trade-off  between  the  measure  of  the  supervised 
sublanguage  and  the  associated  event  disabling  cost  to 
achieve  the  best  performance.  Like  any  other  optimiza¬ 
tion  procedure,  it  is  possible  to  choose  different  perfor¬ 
mance  indices  to  arrive  at  different  optimal  policies 
for  discrete  event  supervisory  control.  It  is  recognized 
that  optimal  control  of  discrete-event  systems  can  be 
achieved  with  a  cost  function  that  may  not  qualify  as 
a  measure  (e.g.,  Sengupta  and  Lafortune  (1998)). 
Nevertheless,  usage  of  a  language  measure  as  the  cost 
function  facilitates  precise  comparative  evaluation  of 
different  supervisors  so  that  the  appropriate  control 
policy(ies)  can  be  conclusively  identified. 

From  the  above  perspectives,  this  paper  presents 
generalizations  of  the  language  measure  (Ray  2005), 
each  generalization  being  a  formal  measure  in  its  own 
right  and  having  physical  implications  that  are  relevant 
to  synthesis  of  discrete-event  supervisory  control 
policies.  These  generalizations  are  achieved  through 
a  new  concept  of  trace  measure  that  is  characterized 
by  both  initiating  and  terminating  states  as  well  as  the 
length  of  the  trace  and  the  choice  of  a  vector 
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norm  (Naylor  and  Sell  1982).  The  concept  of  generaliza¬ 
tion  can  be  viewed  as  renormalization  (Chattopadhyay 
and  Ray  2006)  of  the  (normalized)  language 
measure  (Ray  2005). 

The  paper  is  organized  in  six  sections  including  the 
present  one.  Section  2  briefly  reviews  background  con¬ 
cepts  on  language  measure.  Section  3  derives  measures 
related  to  the  stationary  state  probability  vector  of  the 
finite-state  automaton.  Section  4  introduces  the  notion 
of  shaped  measures,  which  allows  assignment  of  selec¬ 
tive  length-based  importance  to  different  traces  in  the 
generated  language.  It  is  further  shown  that  measures 
introduced  in  §  3  can  be  obtained  as  limits  of  sequences 
of  shaped  measures.  Section  5  presents  an  example  of 
optimal  automaton  configurations.  Section  6  concludes 
the  paper  along  with  recommendations  for  future 
research. 


2.  Brief  review  of  language  measure 

This  section  briefly  reviews  the  concept  of  signed  real 
measure  of  regular  languages  Ray  (2005)  and  Ray 
et  al.  (2005).  Let  G,  =  (£7,  E,  <5,  t/,-,  g,„)  be  a  trim  (i.e., 
accessible  and  co-accessible)  deterministic  finite-state 
automaton  (DFSA)  model  (Ramadge  and  Wonham 
1987)  that  represents  the  discrete-event  dynamics  of  a 
physical  plant,  where  Q  —  {qk-  ■  k  e  Iq}  is  the  set  of 
states  and  Iq  =  {1,2, . . .  ,n)  is  the  index  set  of  states; 
the  automaton  starts  with  the  initial  state  qj,  the  alpha¬ 
bet  of  events  is  E  =  {<t/c  :  k  e  7S},  and  7S  =  {1, 2, . . . ,  1} 
is  the  index  set  of  events;  <5:  Q  x  E  — >•  Q  is  the  (possibly 
partial)  function  of  state  transitions;  and  Qm  = 
■  ■  ■  ,qmr}  is  the  set  of  marked  (i.e., 
accepted)  states  with  qm  —  qj  for  some  j  e  Iq. 

Let  E*  be  the  Kleene  closure  of  E,  i.e.,  the  set  of  all 
finite-length  traces  made  of  the  events  belonging  to  E 
as  well  as  the  empty  trace  e  that  is  viewed  as  the  identity 
of  the  monoid  E*  under  the  operation  of  trace  concate¬ 
nation,  i.e.,  es  —  s  —  se.  The  extension  8*:  Q  x  E*  — ►  Q 
is  defined  recursively  in  the  usual  sense  (Hppcroft 

et  al.  2001). 

Definition  1:  The  language  7,(G,)  generated  by  a  DFSA 
G  initialized  at  the  state  qt  e  Q  is  defined  as 

L(Gi)  =  {s  e  E*  I  8*(qhs)  e  Q}. 

Definition  2:  The  language  L;„(G,)  marked  by  a  DFSA 

Gj  initialized  at  the  state  qt  e  Q  is  defined  as 

Lm(Gi)  =  {s  e  E*  I  8*(qi,s)  e  Qm). 

The  language  L(G,)  is  partitioned  into  non-marked  and 
marked  languages,  7,°(G,)  =  L(G,)  —  Lm(G,)  and  Lm{G,), 
consisting  of  event  traces  that,  starting  from  q,  e  Q, 
terminate  at  one  of  the  non-marked  states  in  Q  —  Qm 
and  one  of  the  marked  states  in  Qm,  respectively. 


The  set  Qm  is  further  partitioned  into  g+  and  Q~, 
where  Q+  contains  all  good  marked  states  that  are 
desired  to  be  terminated  on  and  Q~  contains  all  bad 
marked  states  that  one  may  not  want  to  terminate  on, 
although  it  may  not  always  be  possible  to  avoid  the 
bad  states  while  attempting  to  reach  the  good  states. 
Accordingly,  the  marked  language  Lm(G)  is  further 
partitioned  into  L+(G,)  and  L~n(G,)  consisting  of  good 
and  bad  traces  that,  starting  from  qh  terminate  on  Q+ 
and  Q~,  respectively.  Thus,  the  language  L(  G/)  is  decom¬ 
posed  into  null,  i.e.,  L°{G,).  positive,  i.e.,  L;|?(G,),  and 
negative,  i.e.,  L“(G,)  sublanguages.  A  signed  real  mea¬ 
sure  /r:  2l(g,)  ^ 

[R  =  (— oo,  oo)  is  constructed  for  quan¬ 
titative  evaluation  of  every  event  trace  s  e  L(G,). 

Definition  3:  The  language  of  all  traces  that,  starting  at 
a  state  q ,•  e  Q,  terminates  on  a  state  qj  e  Q,  is  denoted  as 
L{qhqj).  That  is,  L{quqj)  =  e  L(G,)  :  8*(qhs )  =  q,}. 

Definition  4:  The  terminating  characteristic  function 
that  assigns  a  normalized  signed  real  weight  to  state- 
partitioned  sublanguages  L{qi,  <7/),  1=1,2, ...,«, 
j  —  1, 2, ...  ,7i  is  defined  as  x-  Q  — »■  [—  1,  1]  such  that 


[-1,0)  if  <7,  e  Qm 


X/e 


{0} 
(0, 1] 


if  qj  ^  Qm 
lf  qj e  Qm- 


(i) 


Definition  5:  The  event  cost  is  conditioned  on  a  DFSA 
state  at  which  the  event  is  generated,  and  is  defined  as 
it:  L(G,))  x  Q  -»•  [0, 1]  such  that  Wq,  e  Qycrk  e  E, 
Vv  e  L(Gj)) 

(1)  Tt[ok,qj]  =  7 Tjk  e  [0, 1);  Y.k^Jk  <  U 

(2)  jf [cr,  qj\  =  0  if  8(qj,a)  is  undefined;  ir[e,  qj]  =  1; 

(3)  Tt[oks,qj\  =  Tt[ok , qj\  n[s,8(qj,ok)\. 

The  event  cost  matrix  is  defined  as  II y  —  ny  with 
i  e  { 1 , . . . ,  73}  and  j  e  { 1 , . . . ,  773}  where  the  automaton 
has  77  states  and  cardinality  of  the  event  alphabet  E  is  m. 

An  application  of  the  induction  principle  to  part  (3) 
in  Definition  5  shows  ir[st,  qj\  —  ir[s,  qj\7x[t,  8*(qj,  ,s)].  The 
condition  J^k^jk  <  1  provides  a  sufficient  condition 
for  the  existence  of  the  real  signed  measure  (Ray 
2005).  Next  a  measure  of  sublanguages  of  the  plant 
language  L(G,)  is  formulated  in  terms  of  the  signed  char¬ 
acteristic  function  /  and  the  non-negative  event  cost  if. 

Definition  6:  The  state  transition  cost,  jt:  Qx 
Q  —*■  [0, 1),  of  the  DFSA  G,  is  defined  as  follows: 
V<7/,  qj  e  Q, 


Eo-en  q,\,  if  8(qj,  a)  =  qj 
0  if {%/,cr)  =  ?/}  =  0. 
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Consequently,  the  n  x  n  state  transition  cost  Il-matrix 
is  defined  as  II, y  =  jr,y  with  i,  j  e  { 1 , . . . ,  n)  where  the 
number  of  states  in  the  automaton  is  n. 

Although  the  preceding  analysis  reported  in  Ray 
(2005)  and  Ray  et  al.  (2005)  was  intended  for  non- 
probabilistic  regular  languages,  the  event  costs  can 
be  interpreted  as  conditional  probabilities  of  event 
occurrence.  A  brief  discussion  on  the  physical  inter¬ 
pretation  of  the  event  costs  is  given  in  Ray  (2005) 
to  explain  this  issue.  Furthermore,  an  element  i Xjk  of 
the  n-matrix  is  conceptually  similar  to  the  state  tran¬ 
sition  probability  of  a  Markov  chain  or  a  semi- 
Markov  chain  with  the  exception  that  the  equality 
condition  ffk  Ujk  —  1  is  not  satisfied.  Specifically,  the 
inequality  'ffk  Ttjk  <  1,  j  —  \,2, ...  ,n  provides  a  suffi¬ 
cient  condition  for  the  language  measure  to  be  finite. 
This  implies  that  the  preceding  analysis  is  applicable 
to  the  case  of  terminating  probabilistic 
languages  (Garg  1992a,  b)  that  have  a  non-zero  prob¬ 
ability  of  termination  (arising  from  either  intentional 
design  or  unmodelled  dynamics  of  the  plant  automa¬ 
ton)  at  each  state.  If  the  probability  of  termination 
at  each  state,  or  equivalently  the  probability  of  transi¬ 
tion  to  the  (deadlock)  dump  state  from  each  of  the 
other  states  qt  e  Q ,  is  set  identically  equal  to 
0  e  (0, 1),  then  the  Il-matrix  and  the  Il-matrix  can 
be  0-parameterized  as  follows  (Chattopadhyay  and 
Ray  2006): 

n(d)  =  (l-6>)P  and  11(0)  =  ( 1  -  6>)P,  (3) 

where  P  is  the  event  matrix  (also  known  as  the  morph 
matrix),  which  is  derived  from  experimental  data  or 
simulation  data  (Ray  2005)  and  the  resulting  stochastic 
state  transition  matrix  P  is  obtained  from  P  in  a  way 
similar  to  equation  (2).  Since  P  is  a  stochastic  matrix 

(i.e.,  V.P,y=lV/e{l _ ,n}),  the  row  sums 

V.7r,y  =  (1  —  0)  <  1,  j  —  1,2, . . .  ,n  (see  Definition  6) 
make  II  a  contraction  operator  with  the  magnitude  of 
each  of  its  eigenvalues  being  less  than  or  equal  to 
(1—0);  consequently,  [I  —  II]  becomes  invertible  (Ray 
2005). 

In  the  sequel,  the  preceding  measure  construction  is 
generalized  and  the  notion  of  language  measure  is 
extended  to  non-terminating  models  by  first  assuming 
a  uniform  non-zero  probability  of  termination  0  at 
each  state  and  then  computing  the  limit  as  6  ->  0+, 
i.e.,  the  probability  of  termination  approaching  zero. 
The  resulting  0-parameterized  model  coincides  with 
the  desired  non-terminating  model  in  the  limit 
(Chattopadhyay  and  Ray  2006). 

Definition  7:  The  0-parameterized  measure  of  the 
language  L{qt,qj )  is  defined  in  terms  of  its  traces 


(see  Definitions  3,  4  and  5)  as 

AM)  =  7r(.s,  qi)Xj,  Vs  e  L(qh  qfi  (4) 

Aa?u  <?,-))  =  E  /Am)-  (5) 

seL(qi,  qj ) 

Then,  the  measure  of  the  language  L(G,)  of  a  DFSA  G„ 
initialized  at  the  state  qt  e  Q ,  is  defined  as 

/Amc,))  =  EEA(^A)  (6) 

It  is  shown  in  Ray  (2005)  that  the  measure 
(i?  =  fi0(L(Gi))  can  be  expressed  as:  /z®  = 
fZjTty  /z®  +  Xi-  In  vector  notation,  the  0-parameterized 
language  measure  vector  is  expressed  by  making  use  of 
equation  (3)  as 


/  =  [O-(l-0)Pl-1x,  (7) 

where  the  measure  vector  p6  =  [/z®  /z®  •  •  •  /z®]r  and  the 
terminating  characteristic  vector  x  —  [/i  X2  •••  x,,]7. 
Note  that  lime^o+  !*’  of  the  normalized  language 
measure  does  not  exist.  This  problem  has  been  circum¬ 
vented  via  renormalization  (Chattopadhyay  and  Ray 
2006)  as  explained  below. 

The  regular  language  L(G ,)  is  a  sublanguage  of  the 
Kleene  closure  E*  of  the  alphabet  E,  for  which  the  auto¬ 
maton  states  can  be  merged  into  a  single  state.  Then, 
P  degenerates  to  the  1  x  1  identity  matrix  and  the  termi¬ 
nating  characteristic  vector  x  becomes  one-dimensional 
and  can  be  assigned  as  x  —  1  by  normalization. 
Consequently,  the  measure  vector  pe  in  equation  (7) 
degenerates  to  a  scalar  measure  0_1.  The  renormalized 
measure  is  obtained  from  equation  (7)  after  normaliza¬ 
tion  with  respect  to  0_1. 

A,  =  0  [D  —  (1  —  0)P]-'x-  (8) 

3.  Generalization  of  language  measure 

This  section  generalizes  the  notion  of  language  measure 
fie  (see  Definition  7  and  equation  (7)),  which  also  leads 
to  a  renormalized  measure  0®  (see  equation  (8)).  This 
is  achieved  by  redefining  the  measure  of  individual 
traces  in  terms  of  an  initiating  characteristic  function 
f:  Q  i — [0, 1]  that  assigns  a  positive  weight  to  each 
initiating  state  q;  and  serves  as  a  renormalizing  factor 
(i.e.,  a  multiplicative  constant)  for  the  measure  of  the 
traces  initiating  from  the  respective  state.  Figure  1 
illustrates  the  relationship  among  the  initiating  and 
terminating  characteristics.  Different  initiating 


Downloaded  By:  [Pennsylvania  State  University]  At:  17:45  20  April  2007 


792 


I.  Chattopadhyay  and  A.  Ray 


Terminating  characteristic  yj 


Figure  1.  Generalization  of  langauge  measure. 

characteristics  lead  to  different  renormalized 
language  measures  that  may  have  different  physical 
interpretations. 

Definition  8:  The  d-parameterized  generalized  measure 
of  a  singleton  event  trace  set  {5}  c  L(cq,  qj)  c  L(G,)  in 
the  u-algebra  2L(G,1  is  defined  as 

=  &/AM)  -  H(di)x{s,  qi)Xj ,  is  e  L{qu  qj).  (9) 


Lemma  1:  For  p  —  1,  the  initiating  characteristic 

$(0)  =  o.  Hid  e  (0, 1].  (12) 

Proof:  Let  e  =  [1  •  •  •  1] 7 .  Since  P  is  stochastic,  non¬ 
negativity  of  'l'(d)  follows  from  the  following  expansion: 

OO 

'l'(d)  =  J^(l  -  0)kPk  id  e  (0, 1] 

k=0 

OO  OO 

=4,  y(Q)e  =  ^(1  -  8)kPke  =  Jjd  -  e)kg  = 

k= 0  k= 0 

which  implies  that  ||  'iJ(6)jo  ||  1  =  d_1  ii  =>$}  —6.  □ 

The  d-parameterized  generalized  measure  for  p  =  1  is 
obtained  in  the  vector  notation  as 

A  =d[D-(l-d)P]-1x  (13) 


The  generalized  measure  of  L{cjj,  qj)  is  defined  as 

if(L(qi,  qj))  =  J2  AM)-  (10) 

seL(qt,  qj) 

The  generalized  measure  of  a  DFSA  G„  initialized 
at  the  state  qt  e  Q,  is  denoted  as  d®  =  d®(L(G,))  = 
E/  qj)). 

Now  it  is  ascertained  that  Definition  8  satisfies  the 
properties  of  a  measure  on  the  defined  u-algebra. 

Proposition  1:  The  generalized  measure 

pB.  2 Wt)  | - >.  [R> 

is  defined  on  the  measure  space 

(L(Gi),  2l(g‘>,  fj. 

Proof:  It  suffices  to  establish  u-additivity  from  the 
following  fact.  For  a  fixed  d  e  (0, 1),  d®  is  the  product 
of  £,•  (which  is  a  constant)  and  /r®  which  is  a  signed 
real  measure  on  the  u-algebra  2UG‘K  □ 

A  special  family  of  initiating  characteristic  functions 
is  considered  for  the  generalized  language  measure. 

Definition  9:  Let  T(d)  =  [D  —  (1  —  d)P]_1.  The  f/r(amily 
of  initiating  characteristic  functions  is  defined  as 

Ad)  =  ||m.||;1  V/»e[ l,oo],  Vd  e  (0, 1),  (11) 

where  ||  •  ||;;  denotes  the  f^-norm  of  •;  and  /th  row  of  a 
matrix  M  is  denoted  as  Mlt)  and  the  /th  column  as  M0y. 

Remark  1:  Note  that  lim^o4  j’(0)  does  not  exist  due 
to  non-invertibility  of  the  operator  [D  —  P].  However, 
(d_1ff(d))|0=o  is  well-defined  by  virtue  of  norm 
continuity  (Naylor  and  Sell  1982)  and  hence 
limfl^o+  d_1£f($)  exists. 


which  is  identical  to  the  renormalized  measure  in 
equation  (8). 

In  general,  the  d-parameterized  generalized  measure 
for  p  e  [1, 00]  is  obtained  in  the  matrix  notation  as 


m 


ml 


(i4) 


Non-negativity  of  P  and  invertibility  of  [/— (1  —  d)P] 
guarantee  that  ||'I'(d)10||/,  e  (0, 00)  V/,  which  implies 
§f(d)  e  (0, 00)  V  p  e  [1, 00]  V  d  e  (0, 00). 


3.1  Limiting  values  of  i)°\p  as  0  — »•  0+ 

This  section  computes  the  generalized  measures  B®^ 
as  d  — ►  0+,  based  on  the  state  transition  probability 
matrix  P  of  a  stationary  Markov  chain  with  finitely 
many  states.  Then,  P  is  a  stochastic  matrix.  That  is, 
P  is  non-negative  with  each  row  sum  being  identically 
equal  to  unity  (Bapat  and  Raghavan  1997). 

Proposition  2:  For  every  stochastic  matrix  P,  the 
following  limit  exists 


,15) 

7=0 

where  2P  is  a  stochastic  matrix.  Furthermore,  S’ 
commutes  with  P  and  is  idempotent.  That  is, 

P3>  =  3>?  =  9>  =  g>2. 


(16) 
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Proof:  The  proof  is  given  in  Bapat  and  Raghavan 
(1997).  □ 

Since  P  is  a  stochastic  matrix,  [D  -  P]e  =  0  where 
e  =  [l,l,...,l]r.  Therefore,  [I  —  P]  is  not  invertible 
for  any  stochastic  matrix  P;  however,  [D  —  (1  —  0)P]  is 
always  invertible  for  9  e  (0, 1).  The  lemma  to  the  next 
proposition  shows  that  [D  —  P  +  (P]  is  invertible. 

Proposition  3:  The  matrix  [I  —  P  +  a-P]  is  invertible  for 
all  afiO. 

Proof:  The  proof  is  based  on  the  commutative  and 
idempotent  properties  of  (P  in  equation  (16)  and  uses 
the  principle  of  contradiction. 

Let  [D  —  P  +  a.P]  be  non-invertible  for  an 
arbitrary  afiO.  Then,  there  is  a  vector  dfi  0  such  that 

[D  —  P  +  a(P]i)  —  0 

=£•  [P  —  aSfd  —  0  =>  at?[P  —  aSfd  —  aSPd 
=>  [at?  —  of  (Pfi)  —  a(Pi) 

oftPd  —  0  =>•  (P§  —  0  because  a  0. 

Hence,  P0  =  P0  —  uePi)  —  [P  —  a(P\&  —  0 

=>  Pkd  =  0  VA  e  N  U  {0}, 


Proposition  5: 

lim  0[D  —  (1  —  0)P]-1  =  g>,  (18) 

0^0+  v  ' 

Proof:  For  0  e  (0, 1),  it  follows  from  equation  (16) 
and  Lemma  3  that 

0[D-(1  -9) P]~'-(P 

OO  OO 

=  e  J2  ((!  -  °)kpk)  -  o  ^(i  -  ef  &> 

k= 0  k= 0 

OO 

=  0^(1  -9)k{Pk  -  SP) 

k=  0 
oo 

=  0^(1  -  0)A  (P  -  (pf  by  Lemma  3 

k=  0 

=  0[O  -  (1  -  0)(P  -  ,?)]-'  by  Lemma  2 

=>•  lim  (0[D  —  (1  -Q)P]-'-3P\ 

0— »o+  v  ’ 

=  lim  0[O  — (1  -0)(P-  9>)]~\ 

0->-O+ 


which  implies 


Since,  for  continuous  functions  /( ■ )  and  g(  ■ )  with 


—  P^J  0  —  0  VA  ==>■  ^lim  |  P^j  0  =  (Pd  —  0 
=>0  =  0  because  (Pi)  —  0. 


lim  f{9)  —  0  and  lim  g(0)  <  oo 

0^0+  0^0+ 

=4>  lim  f(0)g(9)  =  0, 

0 — >-0+ 


This  is  a  contradiction.  □ 

Lemma  2:  The  matrix  [D  —  P  +  (P\  is  invertible. 

Proof:  The  proof  follows  by  setting  a  — l  in 
Proposition  3.  □ 

Proposition  4: 

[P  -  a(pf—  Pk  -  [1  -  (1  -  a)k](P,  V/c  e  N  Va  0. 

(17) 


it  follows  from  Lemma  2  that 


lim  0[D  —  (1  —  0)(P  —  (P)Y  —  0 

0^0+ 

=>  lim  (0[l  -  (1  -  0)P]_1-,^)  =  0. 

0^0+  ’ 


The  proof  is  thus  complete. 


□ 


Proposition  6:  For  every  stochastic  matrix  P,  the 
generalized  measure  is  expressed  as 


Proof:  The  above  identity  is  valid  for  k  —  0  and  k  —  1 . 

It  is  also  true  for  k  —  2  by  virtue  of  the  commutative 

and  idempotent  properties  of  (P  in  equation  (16).  The  ^°|  =  lim  ,  = 

proof  follows  directly  by  the  method  of  induction.  □ 

Lemma  3:  [P  —  t?]k—  P/f  —  (P  VA'  e  N. 

Proof:  The  proof  follows  by  setting  a  —  1  in 

Proposition  4.  □  where  (?i0  is  the  ith  row  of  (P. 


(19) 
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Proof:  Following  equation  (11)  in  Definition  9,  it 
suffices  to  show  that 


located  within  the  unit  radius  circle  with  center  at 
the  origin. 


~?m  ■■■  o 


lim 

6^0 


m 


[D  _  (1  _  0)P]  1 


0  ■■■  %(0)  J 


'll^loll  p 

o  _ 

—  I 

\\,qp-  II 

II  fJio  || p 

0 

II  dPno  \\p  _ 

s>no 

The  above  identity  is  a  direct  consequence  of  the 
following  two  relations: 


For  a  primitive  stochastic  matrix,  the  expression  for 
in  equation  (19)  of  Proposition  6  is  simplified  as 
presented  in  the  following  proposition. 

Proposition  7:  For  a  primitive  stochastic  matrix  P,  the 
generalized  measure  is  expressed  as 


(20) 


where  pT P  =  pT. 


limdfD  —  (1  -ffiPF1  =  & 

e^o  L 


Proof:  From  the  properties  (i)  and  (ii)  of  primitive 
matrices,  it  follows  that 


■■■  o 

m  ■ 

0  ■■■  U(0)_ 

^Wp  ■■■  o 


^jo  =pTVj<=  {l,...,n},  (21) 

where  pT  is  the  state  probability  vector  of  the  associated 
Markov  chain.  Then,  the  proof  follows  from 
Proposition  6.  □ 


0 


The  first  relation  is  a  restatement  of  equation  (18)  in 
Proposition  6.  The  second  relation  is  obtained  from 
continuity  of  norm  in  equation  (11)  (see  Remark  1).  □ 


We  now  consider  the  special  class  of  primitive 
(i.e.,  irreducible  and  acyclic  (Bapat  and  Raghavan 
1997))  stochastic  matrices.  The  restriction  of  primitivity 
is  valid  for  many  applications  such  as  finite-state 
machines  without  any  deadlock  or  local  livelock.  A 
primitive  stochastic  matrix  P  has  the  following 
properties  (Bapat  and  Raghavan  1997): 

(i)  lim*_H5o  PA  =  9*  and  P.y  =  e/'P  =  g>  =  gp1 

(ii)  The  matrix  3P  has  the  following  structure: 


P 


rq 


SP  = 


P 


T 


where  pT P  =  pT 


3.2  Physical  interpretation  of  the  |  p  measures 

All  entries  of  the  t>(i|p  vector  in  equation  (19)  are 
identical  for  a  primitive  stochastic  matrix  and  hence  a 
single  entry  can  be  taken  as  a  scalar  measure,  i)°\p  = 
pTx/\\p\\P,  of  the  regular  language  of  the  underlying 
automaton.  For  all  p  e  [l,oo],  the  measure  repre¬ 
sents  the  long-range  behaviour  of  the  plant  dynamics 
in  terms  of  the  (assigned)  terminating  characteristics 
and  the  stationary  state  probability  vector  of  the  finite 
Markov  chain  model.  Flowever,  the  measures  for  differ¬ 
ent  values  of  p  are  not  equivalent  in  the  sense  that  a  con¬ 
trol  policy  optimizing  does  not  necessarily  coincide 
with  one  that  optimizes  #°L  for  p  q.  For  example, 
a  control  policy  that  maximizes  selectively  disables 
controllable  events  such  that  pTj_  is  maximized;  and 
a  control  policy  that  maximizes  d°|2  chooses  an  automa¬ 
ton  configuration  to  make  the  stationary  state  probabil¬ 
ity  vector  p  closest  to  the  terminal  characteristic  vector  % 
in  the  Euclidean  sense.  For  physical  understanding 
and  visualization,  let  3>  be  a  bounded  submanifold  of 
such  that 


implying  that  p  is  the  left  eigenvector  of  P 
corresponding  to  its  unique  unity  eigenvalue 

(iii)  Upon  lx -normalization,  p  becomes  the  state 
probability  vector  of  the  stationary  Markov  chain 
associated  with  the  stochastic  primitive  matrix  P. 

(iv)  The  spectral  radius  of  the  matrix  (P  —  Sf)  is  less 
than  unity,  i.e.,  the  eigenvalues  of  (P  —  ^)  are 


Vp=  [p\,...,Pn]  e  9>  with 


Pi  >  0 

£7=o  Pi  =  ll/’H/i  =  L 

(22) 


Then,  for  any  n-state  automaton,  the  stationary  state 
probability  vector  is  p  e  SP.  Figure  2  illustrates  S’  for 
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Figure  2.  Representation  of  the  ./'-plane  for  three  states. 


Lemma  4: 

Let 

v  be 

a  n-dimensional  vector 

with 

vi  e 

[0, 1]  and  J^i 

Vi  —  1 

.  Then  we  have 

(i) 

I  M  /;  £ 

[n(l-pVp,  1] 

dp  e  [l,oo] 

(23) 

(ii) 

VI 

Oh 

3" 

IMI? 

Wp  > 

q  with  p,q  e  [1,  oo] 

(24) 

Proof:  For  Assertion  (i), 

vt  e  [0,1]  =>•  <  Vi  =>•  vf  <  Vi  llvll,  <  1. 

i  i 

The  result  follows  by  noting  that  the  smallest 
value  is  attained  when  all  u,-  are  equal  i.e. 
Vj  —  1  /n  Vi.  Assertion  (ii)  follows  by  noting  that 
vf  <  vf  if  p  >  q.  □ 


77  =  3,  where  the  central  point  pc  denotes  the  uniform 
probability  vector  [\/n, . . . ,  l/?z],  which  is  interpreted 
to  have  maximum  entropy  log2  n  in  the  Shannon 
sense  (Cover  and  Thomas  1991).  Moving  away  from 
pc  on  the  .'/'-plane,  the  distribution  becomes  non- 
uniform,  i.e.,  the  Shannon  entropy  5  = 
—  J2k=\{pk^og2pk)  decreases  toward  zero. 

In  view  of  the  above  discussion  and  Lemma  4,  a 
supervisory  control  policy  can  be  constructed  by  opti¬ 
mizing  tiP\p  in  the  sense  of  equation  (19)  for  a  specified 
/te[l,oo]  to  obtain  a  stationary  state  probability 
vector  p.  For  example,  if  p—l,  then  the  optimization 
algorithm  attempts  to  choose  p  as  a  unit  vector  in  the 
direction  of  one  of  the  axes  of  D?"  for  which  the 
/-vector  has  the  largest  element;  if  this  is  the  case, 
then  Shannon  entropy  S—0.  lip  — 2,  then  the  optimiza¬ 
tion  algorithm  attempts  to  choose  p  as  the  point  of 
intersection  of  the  x  vector  with  the  ./-plane;  in  this 
case,  the  Shannon  entropy  is  S'  >  0  unless  x  is  coincident 
with  one  of  the  axes  of  IR".  For  p  >  2,  the  algorithm 
attempts  to  choose  p  closer  to  the  central  point  pc 
more  and  more  aggressively  as  p  increases  toward 
infinity,  for  which  the  Shannon  entropy  5  increases 
toward  its  maximum  value  log2  n. 

Three  measures  are  considered  to  be  significant;  d°|  | 
and  d°\x  optimal  policies  are  useful  to  obtain  low 
and  high  entropy  (thermodynamically  stable)  distribu¬ 
tions,  respectively,  and  d°|2  optimality  is  useful  when 
the  problem  definition  requires  achieving  a  target 
distribution  over  the  plant  states  as  closely  as  the 
controllability  criteria  would  allow.  An  example  is 
given  in  §  5. 

The  following  lemma  is  useful  for  interpretation 
of  the  D°|p  measures. 


4.  Shaped  measures 

The  measures  defined  in  the  previous  sections  put  equal 
importance  to  all  traces  in  the  generated  language  of 
an  automaton,  including  traces  of  unbounded  lengths. 
This  section  investigates  formal  measures  that  generalize 
the  process  of  assigning  importance  or  weight  to  a 
trace  as  a  function  of  its  length.  For  a  given  automaton 
Gh  a  partition  of  the  generated  language  L(G,)  is 
obtained  as: 


L(Gj)  =  u  3}\  where 


= 


i=0 


a>  eL(qi,qj):  \co\  =rj. 

(25) 


(Note  that  ^n,y.s  3}\  —  0.)  From  u-additivity  of 
the  language  measure  (Ray  2005),  the  following  notion 
of  language  measure  is  introduced. 

Definition  10:  For  a  given  starting  state  qt  and  the 
parameter  0  e  (0, 1),  the  shaped  measure  of  the  language 
L(Gj)  is  defined  as 


v(L{Gj))  =  Y^iAi^d-  (26) 

r=0 

The  above  definition,  as  observed  before,  fails  to  exist 
as  6— >  0+.  The  singularity  at  0  =  0  is  alleviated  by  a 
shaping  sequence  that  provides  appropriate  weights  on 
the  individual  terms  of  the  infinite  sum  in  equation 
(26).  The  next  proposition  establishes  that  every 
fi-sequence  qualifies  as  a  shaping  sequence. 
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Proposition  8:  For  a  real  l  r sequence  F  =  { y,-}  with 
Yi  e  [0,  oo), 

OO 

(i)  £/$**)*  <oc  Vfle(0,l)  (27) 

i=  0 

oo 

(ii)  hm  yi  <  oo.  (28) 

/=0 

Proof:  The  proof  of  the  proposition  requires  the 
following  lemma.  □ 

Lemma  5:  The  following  expression  holds  for  the 
O-parameterized  shaped  measure 

=  (i  -  oyp'x-  (29) 

Proof:  From  Definition  7,  we  have 

lAi-A)  =  E  E  n(qi,(o)Xj 

,=  1  Hqfy) n*J 

=  E{E---E7r**"-;Ty}x/ 

7=1  l  I'l  ir  J 

=  En^  =  E(1-WyX/ 

j  i 

Using  Lemma  5  and  noting  that  for  all  9  e  [0, 1),  it 
follows  that 

11(1  -e/Fxlloc  <  |1  -o\r  IIPII^  llxlloo  <  1,  (30) 

where  ||  •  is  the  induced  sup-norm  of  •.  □ 

The  shaped  measure  is  now  formally  defined  based  on 
Proposition  33  by  setting  the  parameter  0  to  0. 

Definition  11:  Let  T  =  {/,}  be  a  £l -sequence  of  non¬ 
negative  real  numbers  (called  the  shaping  sequence 
in  the  sequel).  The  shaped  measure  rf  of  a  trace  set 
M  £  T(cp ,  qj)  c  L(Gj)  with  j^ieNUjO)  relative  to 
T  is  defined  as 

tf ( W)  =  /AM)fc  =  0(s,  qdx(qj)k  v e  Uau  qj)-  (3 1 ) 

The  shaped  measure  of  L(q,,  qj)  is  defined  as: 

OO 

A {L(qi,  qj))  =  E  E  rr({,v}).  (32) 

r=0  , 

Um.lj)  n< 

The  shaped  measure  of  a  DFSA  Gh  relative  to  the 
sequence  F  and  initialized  at  the  state  qt  e  Q,  is 
denoted  as:  rf  =  rf  (L(Gf)  -  ff  j  <//))• 


The  shaped  measure  vector,  relative  to  the  sequence 
T,  is  denoted  as:  rr  =  [r|’, . . . ,  r'j. 

Remark  2:  If  the  short-term  behaviour  of  the 
discrete-event  system  is  of  interest,  then  all  but  finitely 
many  elements  of  the  shaping  sequence  F  =  {y,}  in 
Definition  11  could  be  restricted  to  be  zeros.  Then, 
there  exists  r*  e  N  such  that  3}\  —  0  Vr  >  r* ,  i.e.,  the 
generated  language  has  only  bounded  length  traces. 

4.1  Relation  between  t/  (/i1  and  it0^  measures 

In  spite  of  a  different  construction,  shaped  measures  are 
related  to  the  generalized  measure  defined  in  §3. 
Specifically,  there  exist  sequences  of  shaped  measures 
that  converge  to  d0^. 

Remark  3:  Let  /?e[l,oo]  and  let  Fi(/;),/ceN  be  a 
sequence  of  non-negative  real  numbers,  whose  all  ele¬ 
ments,  except  the  Arth  one,  are  zeroes  and  the  Arth  element 
is  ||p||  Let  T(p)  =  lim/^oo  F ^(p).  Then,  it  follows  from 
Proposition  6  or  Proposition  7  that  there  exists  r(/?) 
such  that  =  d°]p  V/;  e  [1,  oo]. 

4.2  Physical  interpretation  of  shaped  measures 

A  shaping  sequence  F  specifies  length-based  relative 
importance  of  traces  in  the  generated  language. 
Intuitively,  one  is  rarely  interested  in  all  traces  generated 
by  an  automaton.  More  often  than  not,  either  short 
traces  or  very  long  traces  (specifically  of  unbounded 
length)  are  important.  The  first  case  is  handled  by  shap¬ 
ing  sequences  with  finitely  many  non-zero  terms  and 
the  latter,  shown  in  Remark  3  is  viewed  as  a  limit  of 
the  shaped  measures.  However,  shaping  sequences  can 
be  more  complicated;  the  only  requirement  is  that  the 
sequence  be  in  t\  (see  Proposition  8).  In  this  context, 
Remark  3  implies  that  addresses  the  long-term 
behaviour  of  the  discrete-event  system  based  on  the 
traces  of  unbounded  length  with  no  importance  to 
finite  traces.  This  follows  from  the  fact  that,  for  p  —  1 
and  all  elements  of  the  sequence  F/c  are  zeros  with 
the  exception  of  the  k,h  element  being  equal  to  1. 


5.  An  illustrative  example 

Figure  3  shows  the  finite-state  automaton  model  of 
the  plant,  where  the  state  set  Q  —  {q\, . .  ■ ,  <79}  and  the 
event  alphabet  £  =  {crr,  07,  of ,  oh ,  op,  orf,  orb,  olh ,  an}. 
The  transitions,  shown  by  dashed  lines,  are  controllable 
and  those,  shown  by  solid  lines,  are  uncontrollable.  The 
state  transition  matrix  P  is  given  in  table  1 .  The  stochas¬ 
tic  matrix  P  is  primitive  because  P2  is  a  positive  matrix. 
The  stationary  state  probability  vector  of  P  and  the 
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/-vector  for  the  DFSA  are  made  as  they  are  familiar  norms  used  in  engineering 

pr=  [0.234  0.041  0.065  0.084  0.095  0.016  0.153  0.147  0.076] 

/  =  [0.66  -0.42  -0.97  0.52  -0.49  -0.57  0.57  -0.09  0.43]'. 


The  scalar  measures  d°\p  and  ||p||p  for  the  primitive 
matrix  in  table  1  are  plotted  for  different  values  of  p 
in  figure  4.  A  shaping  sequence  T  evaluates  the  short¬ 
term  behaviour  based  on  traces  of  length  less  than 
60  and  the  resulting  measure  vector  is 

rr  =  [0.276  -0.252  0.627  -0.375 

Supervisory  control  policies  have  been  computed 
based  on  Hpl^  and  rr  by  optimizing  the  respective 
scalar  measures  via  a  standard  search  algorithm. 
Different  values  of  p  —  1,4,  and  oo  are  chosen  to  illus¬ 
trate  the  fact  that  they  result  in  different  optimal 
control  policies.  The  choices  of  p  —  1  and  p  —  oo  are 


analysis;  and  the  choice  of  p  —  4  is  made  because 
it  effects  are  intermediate  between  p  —  1  and  p  —  oo 
and  are  different  from  those  of  the  Euclidean  norm 
p  —  2  (see  §3).  For  these  cases,  the  results  of 
improved  performance  under  optimal  supervision  are 

0.256  -0.055  -0.059  -0.103  0.475  ]r. 
summarized  below. 

For p—  1,  i?°||  is  increased  from  0.12  to  0.35. 

For  p  =  A ,  i?°|4  is  increased  from  0.48  to  1.03. 

For  p  —  oo,  ?7°  Iqq  is  increased  from  0.52  to  1.3  and 
rr  is  increased  elementwise  from 


Figure  3.  Plant  automaton  model. 


Figure  4.  Profiles  of  and  \\p\\p  with  p.  Note  the 

stabilizing  feature  of  each  plot  for  increasing  p. 


Table  1.  State  transition  matrix  P  of  the  plant  automaton  model. 


0 

0.015 

0.102 

0.041 

0.120 

0.048 

0.300 

0.139 

0.139 

0.372 

0 

0.131 

0 

0 

0 

0 

0 

0 

0.130 

0.319 

0 

0.551 

0 

0 

0 

0 

0 

0.087 

0 

0.424 

0 

0.489 

0 

0 

0 

0 

0.351 

0 

0 

0.411 

0 

0.238 

0 

0 

0 

0.337 

0 

0 

0 

0.240 

0 

0.423 

0 

0 

0.069 

0 

0 

0 

0 

0.470 

0 

0.460 

0.460 

0.738 

0 

0 

0 

0 

0 

0.259 

0 

0 

0.199 

0.218 

0 

0 

0 

0 

0 

0.583 

0.583 
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Table  2  Optimal  decision  for  disabling  of  controllable  events. 


Controllable 

events 

A 

A 

1>°[ 
u  loo 

Tr 

9l%95 

X 

X 

X 

X 

n  si 

9l  ->•  96 

X 

X 

X 

X 

9l  %  <79 

/ 

/ 

/ 

/ 

92%9i 

/ 

/ 

/ 

/ 

<72  S  93 

X 

X 

X 

X 

92  %  99 

/ 

X 

/ 

X 

93%9i 

/ 

/ 

/ 

/ 

9s%  94 

/ 

/ 

/ 

/ 

95%  96 

/ 

/ 

/ 

/ 

97%9i 

X 

X 

X 

/ 

97%  96 

X 

X 

X 

X 

99  %  98 

X 

X 

X 

X 

Of 

#9  ->  Ql 

X 

X 

/ 

X 

[1.2 

1.5 

0.9 

0.3 

1.1 

1.4 

1.2 

1.7 

1.4]r 

to 

[3.4 

bo 

2.5 

1.7 

2.6 

3.3 

3.5 

3.7 

4.0]r. 

Table  2  enumerates  the  optimal  decision  sets  for 
disabling  of  controllable  events  obtained  in  the  above 
four  cases,  where  x  and  /  indicate  disabled  controllable 
events  and  enabled  controllable  events,  respectively.  The 
decisions  are  made  from  the  stationary  state  probability 
distributions  achieved  from  the  optimal  policies  shown 
in  figure  5.  It  is  seen  that  the  i?°|  l -optimal  policy  achieves 
both  maximum  and  minimum  probability  values  in 
states  9  and  6,  respectively.  This  shows  that  77°  |, -optimal 
policy  does  indeed  produce  relatively  less  uniform  distri¬ 
bution  in  comparison  to  the  -optimal  policy,  as 
stated  in  §3.  It  is  also  noted  that  the  rr-optimal  policy 
achieves  the  least  uniform  distribution. 


6.  Summary,  conclusions,  and  future  work 

This  paper  formulates  and  validates  a  concept  of  gener¬ 
alization  of  signed  real  measure  of  regular  languages, 
which  also  leads  to  renormalization  (Chattopadhyay 
and  Ray  2006)  of  the  normalized  measure  and  eliminates 
the  need  for  a  user-selectable  parameter  in  the  original 
concept  of  language  measure  Ray  (2005).  These  general¬ 
izations  are  achieved  through  a  trace  measure  that  is 
characterized  by  both  initial  and  terminal  states  as  well 
as  the  length  of  the  trace  and  the  choice  of  a  vector 
norm  for  renormalization.  The  generalized  measures 
with  different  norms  are  not  equivalent  in  the  sense 
that  the  respective  optimal  control  policies  with  these 
measures  as  the  performance  cost  functionals  are  differ¬ 
ent.  These  concepts  are  illustrated  with  simple  examples 
for  quantitative  analysis  and  synthesis  of  discrete-event 


- <-  States 

Figure  5.  Stable  distributions  for  computed  optimal 
policies. 

supervisory  control  systems.  It  is  envisioned  that 
optimal  supervisory  decision  &  control  of  discrete- 
event  systems  (Sengupta  and  Lafortune  1998,  Ray  et 
al.  2004)  can  be  enhanced  through  appropriate  selection 
of  a  language  measure  to  enhance  the  objectives  at  hand. 
In  this  context,  future  research  is  recommended  in  the 
following  areas: 

•  Generalization  of  the  language-measure-based  opti¬ 
mal  control  algorithms  (Ray  et  al.  2004)  for  sub¬ 
stochastic  transition  matrices  to  the  stochastic  case. 
A  potential  application  is  to  compute  a  sufficiently 
small  termination  probability  0  such  that,  as  0  -»  0+, 
the  optimal  control  policies  approach  the  true  situa¬ 
tion  for  the  non-terminating  plant. 

•  Extension  the  concept  of  (regular)  language  measure 
for  (non-regular)  languages  higher  up  in  the 
Chomsky  Hierarchy  such  as  context-free  and  context 
-sensitive  languages.  A  first  attempt  to  extend  the 
concept  of  the  language  measure  to  linear  grammars 
was  reported  in  (Ray  et  al.  2004).  Further  investiga¬ 
tions  in  this  direction  is  required  for  extension  of  the 
concept  to  more  complex  models. 

•  Applications  of  language  measure  in  anomaly  detec¬ 
tion,  model  identification  and  order  reduction,  and 
construction  of  interfaces  between  continuously 
varying  and  discrete-event  spaces. 
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